Booting Linux Faster 625
krony writes "IBM's DeveloperWorks explains how to decrease boot times for your Linux box. The concept is to load system services in parallel when possible. Most surprising to me is the use of 'make' to handle dependencies between services." The example system shown is able to cut its boot time in half, but the article stresses the effectiveness can vary widly from machine to machine.
Hmm (Score:2, Interesting)
I guess someone has a use for this, or they wouldn't have spent the time working on it. But I don't see it.
I never noticed Linux taking very long to load, and even if it did I doubt I would care very much, as reboots are so rare anyway.
Dual (or more) cpus (Score:2, Interesting)
Even on an HT-enabled P4 this would be cool. Although the I/O would be the limiting factor in the process startup speed, letting multiple proceses start up at once would allow the cpu to switch to others while I/O is being services, much like make -j(# of cpus+1).
Re:Another way to speed up booting Linux (Score:3, Interesting)
Isn't there a way (Score:3, Interesting)
I remember reading about it somewhere, but it was skimpy on details, sufficing to say that it was a "bad idea".
Re:Hmm (Score:5, Interesting)
Do I want a faster boot?
You bet your ass I do.
PowerPC Linux users had compiled boot 'scripts' (Score:5, Interesting)
Boot times went from about 2 minutes, to 35 seconds.
(It took "so long" because it was an old PPC 601 60MHz or something like that).
Distributions such as Mandrake and Gentoo claim they go the extra mile for "performance". I've wondered why neither has cleaned up their boot process.
You wouldn't think Bash is slow from interactive use, but it really it. Piggyback on that speed problem that too many "functions" (OK, *commands*) are standalone executables... greate sub-process, collect result, destroy, rinse repeat.
This is pretty interesting stuff, and I applaud this guys efforts. INIT script achitecture is pretty thankless stuff..
Re:Other things to speed up boot time (Score:2, Interesting)
If the second reason is your main reason to boot quickly, you'll probably want to start X too every time you boot. So waiting to start X until a user says X should be started is no option. Your other suggestions are spot on.
If you'd like to take away the last, big, bottleneck, it would be a good idea to start X in parallel with the other, independant, services. This is exactly what's described in the insightful IBM article. Hooray!
Parallel startup implemented in Mac OS X (Score:5, Interesting)
The big question is "how do you specify dependencies?" The article uses makefiles. In Mac OS X, each startup item has a properties file (associative array) that names the item and specifies all the items that it depends on (http://www.usenix.org/events/bsdcon02/full_paper
Re:Make? (Score:5, Interesting)
For boot you would tell make this:
sshd: network
rpcd: network
But for shutdown you need to tell it this:
network: sshd rpcd
Ideally one set of input data should take care of both cases.
Re:Make? (Score:3, Interesting)
This isn't make's intended use (it was designed for programming), so it's a bit suprising to see it used this way at first.
That said, it does make perfect sense.
Re:Make? (Score:5, Interesting)
> Really? That's an odd statement. How surprising that they choose to use an open-source software application that is designed to compactly represent dependencies for representing dependencies.
Actually, I also found it surprising, and I think I know "make" pretty well. The thing about make is that in 95% of cases almost all of the rules correspond to an actual target file that should be generated or not based on presence and timestamp. There are exceptions, like the usual "all" rule that's called a phony rule since it generates no file. (And make sure you have a ".PHONY: all" line right before it or "touch all" will break your build.) It's usually just there for the dependencies on a bunch of real targets, so you don't have to type "make this && make that && make ...".
Parts of make that they're not using here:
- logic for checking if a real target is up-to-date
- rules for creating specific targets from generic ones, like the
.c.o target
- variable substitutions
- a lot of other things...look at the man/info pages; modern versions of make have a lot of functionality that makes no sense here
And they are using:When I say the syntax doesn't make sense here, I mean (in addition to the usual make complaints) that it's all in one file. Distributors (notably RedHat in particular) have been very serious about separating out stuff into .d directories so that packages don't need to touch each others' files.
So, I think make is the wrong tool for the job here, at least in the long term. A simple tool with separate files for each service would be a win. I don't think the author of the article really cares about that (it's just a little tip for intermediate users), but if a distribution wanted to implement this idea and maintain it, they wouldn't use make.
Re:Just turn off services you don't need (Score:4, Interesting)
Ordinary users, and even many geeks, don't have time to figure out what every service does and whether they use it. A policy of aggresively turning off services (mostly for security, partly for boot time) carries a risk of turning of a service that is needed.
I suggest that there should be a standard framework for dealing with "a needed service is not running" problems. On a desktop Linux, this should pop up a window explaining what service wasn't running, and giving options to do nothing, start the service on a one-time basis, or add the service to boot time start-up (and prompting for root password as required.)
(There can be extra options - don't start the service, and never ask me again. Don't start the service, and never ask me again if this particular program complains about it.)
Re:PowerPC Linux users had compiled boot 'scripts' (Score:5, Interesting)
I use a bunch of homemade Xterminals made out of Nforce boards and we have replaced
No shutdown script is necessary because Xterminal users simply logout and turn them off.
I think one of the biggest slowdowns on PCs is the lame PCBIOS which takes a very long time to run through all the hardware. I remember following LinuxBIOS [linuxbios.org] development. It is so fast, that it was finished checking the computer's hardware before the disk drives finished spinning up.
I got one down to about 3 seconds. (Score:3, Interesting)
My best times were power on to init in about 2.7 seconds. By the time we got the "authentication code" and what not in it was closer to 30 seconds.
Take all that BIOS stuff out and create a truely lean and mean setup with minimal init scripts and you can blaze. Longest step was copying the kernel from slow-mo flash memory in to RAM...
Re:This is actually important (Score:2, Interesting)
I would even start to apply this to desktop machines - just suspend it, don't turn it off all the way.
Cheers.
Andre
Re:Predicted response (Score:5, Interesting)
I think this "problem" you mention is some sort of urban legend. I have heard this same argument countless times, but I have never actually seen this happen. I have been a subscriber to a few Linux mailing lists for several years now, and I have never actually seen someone post "RTFM" as an answer to a question.
Myself, I try to sort of evaluate the person who is asking for help. If I think he has an adventurous soul, and is willing to go through a lot of documentation, I try to orient him to the relevant how-to's. In the other hand, if I feel the person is somewhat impatient, I recommend a Mandrake installation, since it's the most likely to get the user safely past the most annoying problems with minimum fuss.
Re:Predicted Predicted response (Score:3, Interesting)
Re:Predicted Predicted response (Score:3, Interesting)
New boot system (Score:3, Interesting)
*nix needs a major boot/shutdown system upgrade. I have migrated to minit [www.fefe.de], but that is primarily for low memory usage. It allows a rudimentary mechanism for specifying dependencies, but is geared mostly to be minimalistic. This 2003, I think we can come up with something better than Sys V init.
Features of a next gen boot/shutdown service manager:
* uses real dependency traversal on startup and shutdown (maybe using a small theorem prover like CML2, or maybe something like make)
* allows configuration of arbitrary and unlimited sets of services, which can be named by arbitrary string literals - no longer chained to 7 numeric choices. e.g. "roaming laptop", "docked server", "minimal services", etc.
* built-in service start/stop/restart/status/enable/disable tools, and standard service API with bindings for various languages (what, native services? imagine that...we do so for Windows NT+, e.g. apache) as well as Plain Old Shell Scripts. So every freakin' flavor/distro of *nix doesn't have its own fscking way to start/stop/enable/disable services.
A lot of the garbage that goes on during startup (have you looked at the standard redhat scripts?) mounting drives and file systems, setting network and hardware parameters, etc., could probably use being standardized also, and either pulled into drivers or services or something, in a standardized fashion. Ideally all these APIs could be exposed both through command line tools, but also through desktop-integrated GUI tools, so that modifications don't entail digging up some ad hoc script on disk and modifying it and hoping you remember what the fuck you did a year ago in some system script.
Re:Hmm (Score:2, Interesting)
rm it.
If you can stand it, add "noatime" to /etc/fstab
for your Linux partitions. You will always
have 'new mail' mail instead of just 'mail', but
if things are just reading from the drive, the
drive can actually spin down instead of having
to write just to say what it read.
Re:Hmm (Score:1, Interesting)
have you tried removing unused services? apache-perl-php seems to be a default install option (8.3, earlier?), usually "hwscan" (on boot) is done every time (how often do you change hardware in a notebook; this is NOT pcmcia stuff); DMA enabled on disks?; optimized kernel?; have you tried suspend-to-disk|memory?
how much time did you invest in learning (and thus optimizing) your system for your needs? linux (meaning the kernel and other drivers) are rather tolerant concerning the hardware which is THE OPPOSITE of speed-optimization. given the skills and motivation you might try to google for patches for certain chips and apply them. beware that one might use near infinite time on this task (try a lfs install, or use gentoo stage 1). this is (as always) a question of balance, for developers, distro makers and end users.
occasionally forced to dual boot
when i do boot into this 'other' os, i rather think of choosing to do such; seems nicer that way *g*
concerning KDE: this old news; yes, one might call it slow.
gnome maybe faster, or it may not. read, try, learn; there is no alternative (resistance is futile
if you really want, try running a windowmanger without a desktop solution (afterstep?).
if you think you have learned your linux beyond the world of yast(2), switch to another distro (as SuSeconfig is quite good at invalidating usermade changes). if you are bold, try a real os (*bsd).
remember that you don't always get what you want, or at least not for the price you are willing to pay.
have fun!
My own rc/init scripts (Score:5, Interesting)
Back in 1999 I rewrote all the init scripts entirely from scratch. I did this after having spent a few years before hacking at init scripts in BSD/OS, OpenBSD, Redhat, Slackware, and Solaris. I experienced all the crankiness of these systems (Redhat and Solaris were the worst) and this time decided to avoid all that. I gave the scripts entirely different names so as not to conflict with existing scripts (was Slackware at this time). That way I could switch between them with just a change of /etc/inittab. It took a few hours, but I had a running fully functional system by the end of the day, and have been running on those scripts, as subsequently better debugged and tweaked, ever since. They booted up noticeably faster than even the Slackware scripts (which were about as fast as the OpenBSD scripts).
Irontically, I didn't do this to get the boot speed. The init scripts are fast enough now that the kernel initialization time is longer, anyway. What I did this for was because I hated having a bunch of separate directories with symlinks in them for each run level. I didn't like having to use specialized tools to manipulate the system (I wanted to routinely use the tools I would have available if I were running from a rescue floppy trying to fix it). That meant doing things with a basic set of shell commands. Yet I didn't want to abandon having separate scripts for each service/daemon being started (or stopped as the case may be). What I ended up doing was creating a single subdirectory for all the individual service scripts, and making the script name have a pattern that included both the startup sequence (stop sequence simply ran backwards), as well as the run levels. Here's what the names in /etc/sys on my system look like:
Figuring out which run level each service starts in is left as an exercise for the reader. BTW, I think most of the speed comes from the fact that I didn't add a lot of fat to my script system. That's easier to do when you do your own design.
Re:Predicted Predicted response (Score:2, Interesting)
RE:Gentoo parallel startup (Score:3, Interesting)
Disk seeks are the real culprit? (Score:2, Interesting)
The reason this worked was that starting processes in parallel increased disk contention and the extra seeks brought the machine to a crawl.
Sure, that was on a system with very slow seek times, and not much in the way of disk caching and scheduling (an Amiga 500 with no HD). A lot of things have improved since then, but seeks are still extremely slow in machine terms, and we also have virtual memory and demand page loading which I imagine don't help the problem.
Maybe keeping data needed at startup centralised on the disk (e.g. in the boot partition) would help.
As disk accesses during startup are probably pretty predictable (consistent from one boot to the next), it may be possible to pre-load the disk cache to improve startup times.
A simple approach would be to log disk blocks accessed during startup, and then read them (in a sensible order of course, and in parallel across disks) at the start of the next boot.
LinuxBIOS (Score:3, Interesting)
I thought there is an outfit called "linuxbios" that supposed to make re-booting, especially cold-booting a very fast process.
Can anyone here tell me the recent progress of "linuxbios" ?
Thank you !
Re:Predicted response (Score:1, Interesting)
Anyways, my point is: don't assume