How Kernel Hackers Boosted the Speed of Desktop Linux 380
chromatic writes "Kernel hackers Arjan van de Ven and Auke Kok showed off Linux booting in five seconds at last month's Linux Plumbers Conference. Arjan and other hackers have already improved the Linux user experience by reducing power consumption and latency. O'Reilly News interviewed him about his work on improving the Linux experience with PowerTOP, LatencyTOP, and Five-Second Boot."
Re:Does it matter? (Score:4, Informative)
Yes.
Re:Does it matter? (Score:3, Informative)
I'm one of the less lucky ones that only have half of hibernation working, the "resume" part fails.
In case Oreilly goes down again... (Score:4, Informative)
O'Reilly News recently interviewed Arjan van de Ven [fenrus.org] about his efforts to improve Linux performance and reduce power consumption. Arjan works for Intel in the Open Source Technology Center [intel.com]. This interview is approximately 30 minutes.
One of the projects you're probably most known for in the past couple of years is the PowerTOP [lesswatts.org] utility, which I found very fascinating. Looking at some of the gains you've made over the past 18 months, it seems like Linux-based devices are saving a lot more power than they used to. What do you consider the big successes in the past year and a half?
To be honest we fixed effectively the entire Linux desktop space. It's not--PowerTOP is more--it's not just what we fixed with PowerTOP is not individual pieces. We fixed everything. For me that was a success.
Is that everything in terms of not just desktop but servers as well?
Yeah; we fixed not just Evolution. We fixed Firefox; the thing with Firefox was that it wasn't one thing that was broken. Everything had problems and we had to fix all of it. So for me the success was how quickly everything got fixed; it was just amazing.
In this context you consider fixed--everything is no longer broken in the same way or--?
Everything is no longer keeping the CPU out of idle basically.
Do you have a reference machine? I guess I'm asking what's your benchmark for this, a particular software configuration stack or particular type of machine, or are you willing to say it's pretty much every Linux based machine out there?
I'm looking at several machines--my own laptop but to be honest, what runs on my own laptop is what I care about most. At least that's where I got more battery life, this is where I see the changes. I tend to run a quite rich environment on my laptop but I also look at service. We look at all kinds of machines and we see the same trend everywhere in that all the various pieces of it--never polling or keeping the CPU up. They all got fixed.
In fixing this, is there a component of education, for example, saying "Instead of doing a busy wait on a select loop or continually polling you should set a kernel timer and wait for that to call you"?
That's part of it but the biggest thing is that you had no visibility. Just two days ago at IDF I spoke with a developer of the GNOME desktop and he said, yeah; when I saw it happen I fixed it in 10 minutes, but you don't know it's there until you see it from PowerTOP. Adding the visibility turns out to be enough for people to start fixing it. They know how to fix--how to not poll most of the time.
You can't fix something you can't measure.
If you don't see that it happens you don't know it happens and you can't fix it.
Are you getting the same sort of results from other projects you run into?
GNOME was there but it's almost everybody goes oh yeah; we should have not done that; either they fix it themselves or some--a lot of people give them the fix and in general it's like oh yeah; we shouldn't have done that. Unless you see what's happening you don't know what to fix, so the biggest thing that PowerTOP did was add visibility. We can see under the hood what's going on and then we can fix it. And quite often the fix is very simple.
It sounds then, maybe I should be able to say that just about everybody is happy to see this. Is that the case?
Yes; people--all the developers I've worked with--and that's quite a few--they all go oh yeah. Thank you for the fix; we should have no problems in the first place. We didn't know this; it's fixed now. In the beginning I did most of the fixing when PowerTOP was very new and now days the people do it themselves. The developers learn
Re:So is anyone making a distro around these ideas (Score:1, Informative)
You don't want something heavy like Ubuntu on that thing.
Re:Does it matter? (Score:2, Informative)
I see a lot of comments on the LWN article of people talking about starting services after the user sees the desktop as cheating.
Funny, but that's exactly how XP and Vista achieve their 'fast' boot times. XP doesn't actually boot much faster than 2000, once you realize that the services are all still loading in the background.
But, yes, it does matter, laptop or not. Most desktop users don't leave their machines on 24x7. Just because you and I do doesn't, at all, mean that is typical usage.
Re:Should lead to possibly great advertisements (Score:5, Informative)
but if this can be applied generically to most distributions then it should present an excellent opportunity for advertisement.
Not going to happen. If you read the article, you'll see that they compiled all drivers directly into the kernel, so it is essentially an embedded device now. Also consider the fact that they are using a SSD, which is going to decrease boot times regardless of any boot-process improvements.
So basically, you could never apply these speed increases to a generic distro.
Re:So is anyone making a distro around these ideas (Score:3, Informative)
TinyME [tinymelinux.com] (my favorite) which is based on PCLinuxOS which is based on Mandriva
Vectorlinux [vectorlinux.com] based on Slackware
Mepis [mepis.org] based on Debian
Or just sudo apt-get xubuntu-desktop
Re:Let's head some comments off (Score:5, Informative)
We've sent a patch to Greg KH making USB initialization go in parallel which reduces usb initialization from [N * 0.1] seconds (where N is the number of usb ports in your system to [0.1]. This patch is currently in linux-next afaik.
I'm wondering why you would even have PCIe HP turned on on an asus 900 :)
Re:Should lead to possibly great advertisements (Score:5, Informative)
actually we do bring up the network daemon (either connmand or network manager) as soon as we can, but we don't stop the entire startup process.
on my test system here it runs dhclient about when X starts up and the network card receives a reply with a few seconds from that, long before I can start firefox :)
Re:Boots in 5 seconds... (Score:5, Informative)
it's just a normal linux distro. for the demo we used an auto-login but you can have it start to a password screen as well, so it's just as secure as your base distro is....
Huh? (Score:3, Informative)
I've administrated network authenticated openSUSE machines and they definitely benefited from booting faster (compared to older versions of openSUSE) - after all the sooner the kernel finishes the sooner you can start waiting for that DHCP lease...The key is that the moment someone says they want to run NIS/LDAP/NFS you just say "start everything that doesn't depend on the network while you wait for the network to come up". In your case NIS/NFS/autofs/xdm DO need the network so they have to wait until that DHCP lease is acquired. No functionality need be lost but the dependencies/order of certain events need to be maintained (this is what tools like Upstart [ubuntu.com] are about).
Strangely enough in one of the article's comments [lwn.net] you'll find that Arjan isn't advocating a parallel boot:
Ultimately I doubt people are advocating all of this work for your typical network workstation. For starters such systems don't tend to be using solid state disks with unattended login...
Re:Should lead to possibly great advertisements (Score:5, Informative)
we don't even need to recompile the kernel, we can just link the modules in as needed. Something most older UNIX systems actually did.
Re:Does it matter? (Score:4, Informative)
Linux has always tried to be an excellent choice for a very broad range of uses. Just because you do not need fast boots does not nobody else does.
Re:Should lead to possibly great advertisements (Score:3, Informative)
You still get group policies assigned, whether you are in a domain or not.
So yes. You need the group policy service.
Re:Important details I'm not seeing (Score:5, Informative)
The 5-second demo video is of an asus eee pc 901, however the same image boots on an acer aspire 1 also in exactly 5 seconds.
These systems have 512mb of memory, a slow consumer-grade SSD (we used a 4gb partition) and a 1.6GHz atom processor (single core, HT enabled).
this is a _slow_ system compared to any desktop-grade system currently on the shelves.
Re:Important details I'm not seeing (Score:4, Informative)
and we used the 'stock' SSDs in these systems. from the bootcharts you can see that our SSDs top out at 25-30mb/sec throughput on read, far below what most regular hard discs or server-grade ssd's can do.
Re:Should lead to possibly great advertisements (Score:4, Informative)
Nope. Kudos is Greek and singular, just like pathos :)
Re:"Pretty Quick"? Drug of Choice? (Score:4, Informative)
There is no such thing as end-user-OS-boot-time. It depends a lot on device drivers and system background utilities. For example, some piece of hardware AND some release version of its driver maybe causing your trouble... especially if that hardware is removed and the driver probes a lot of time just to be sure. Same for the AV software doing weird things in order to "secure" the system *before* user interaction... At least in the hardware side, this apply for Linux too.
Re:On an old Pentium III laptop... (Score:3, Informative)
Do him a favor... Install Win95B, Win98, or W2K.
Re:Should lead to possibly great advertisements (Score:5, Informative)
A link to the article is here. [ibm.com]
Bear in mind, this is circa 2003. 8)
Kernel modesetting is slow arriving (Score:4, Informative)
Kernel modesetting is kinda here for Intel and ATI graphics cards [phoronix.com] (in Fedora) but it's still stabilizing.
So the answer to your first question is: soon you will be able to set the final video mode in the kernel. As for your second question, doing it in X is not the best solution (as doing it in the kernel means less flickering when X starts, the ability to support graphical kernel panics and nicer virtual terminal switching).
Re:Should lead to possibly great advertisements (Score:2, Informative)
Just a peace of advice...
If you're really serious about being notified of S.M.A.R.T. disk attribute change/selftest failure, you should consider using the -m switch in your /etc/smartd.conf file. The -M switch controls under what conditions it will email. I have mine set to '-M test' so it sends an email at boot up to ensure that it can mail notifications successfully and also serves to let me know when the server has arbitrarily rebooted which rarely happens when you consider 30 to 90 day uptimes on average. I do this precisely on Fedora (and CentOS and RedHat Ent. can do the same just without the temp monitoring because their packaged version is too old and doesn't have the temp monitoring feature for [S/P]ATA drives). I use it to notify me via email when something happens to the hard drive instantly, like temperature has gone too high, or God forbid a regularly scheduled disk test failed (hasn't happened yet). I used to have my personal headless fileserver email me to my gmail account which I got instant notification of on my Blackberry...ever since my ISP put themselves (the IP subnet I'm on) on a SPAM no mail list...gmail now refuses my mail from my box so now I just email to root and redirect all of that to my primary user account and just run a simple local mailbox poll program to notify me at the desktop when new mail arrives. I hope to come up with something in the near future for better instant notification. But otherwise it works great for logwatch and if it wasn't for the ISP intentionally blacklisting their customer IP space I would still get instant notification of smartd events which are rare so don't worry about them filling up your inbox. I get one lately only when the system reboots and a test message is issued. Only once this year did I get a real warning that the temp had gone over 50 degrees C while I was at work...turned out the A/C had been shutoff the whole day during the summer and I was somewhat overdue to clean out some dust in the fans.
#man smartd.conf can prove to be a useful read, and if you are using Fedora, alot of the switches and options are actually documented right in the /etc/smartd.conf file. Its a self documenting config file.
Sly
Re:Should lead to possibly great advertisements (Score:5, Informative)
32-bit operating systems on x86 can get around this with PAE. Windows XP and Vista don't have support for it, though.
http://www.appleinsider.com/articles/08/09/03/road_to_mac_os_x_snow_leopard_64_bits_santa_rosa_and_the_great_pc_swindle.html [appleinsider.com]
The half gig "lost" is because x86 is a memory-mapped architecture. The graphics card memory and all your peripherals are mapped over the top part of your memory address space.
Re:Should lead to possibly great advertisements (Score:5, Informative)
You obviously fail it.
The 32 bit limitation is 4 gigs of RAM (total).
Your video card's memory is mapped into the same address space as your system ram, but from the top of the address space downward, rather than system ram (mapped from bottom of address space upward).
The video ram has priority, thus if you have 3G of ram and a 1G video card you are using 100% of your address space. any more system memory installed will be masked by the video ram and thus is "lost".
-nB
Re:Should lead to possibly great advertisements (Score:3, Informative)
The original reported problem of Windows only reporting 3 GB of RAM is probably related to one of the many problems described in this Microsoft article: http://www.microsoft.com/whdc/system/platform/server/PAE/PAEdrv.mspx [microsoft.com]
It doesn't mention Vista specifically, but it does list different physical address space limitation for the various editions of other Windows versions. Among others, XP is mentioned as being limited to 4 GB of physical address space, and I'd guess the cause of that would be its not being licensed for more than that. By similar reasoning, it wouldn't surprise me if most editions of Vista have the same limitation. I guess that's what you get for using evilware.
Re:Should lead to possibly great advertisements (Score:1, Informative)
It's more like 3.5. Note that the hard limit for all 32-bit operating systems is 4GB. Windows manages to lose half a gig somewhere.
I guess Linux is better than all 32-bit operating systems then. It handles 64 GB RAM on IA32, since the Pentium Pro or thereabouts.
32 bits is the size of the registers, not the size of the address bus. A pentium 4 is not limited to 4 GB RAM, just as the old 16-bit PC's weren't limited to 64 GB (640 KB should be enough for anyone, remember?), and the Commodore 64 isn't limited to 256 bytes.