The ~200 Line Linux Kernel Patch That Does Wonders 603
An anonymous reader writes "There is a relatively miniscule patch to the Linux kernel scheduler being queued up for Linux 2.6.38 that is proving to have dramatic results for those multi-tasking on the desktop. Phoronix is reporting the ~200 line Linux kernel patch that does wonders with before and after videos demonstrating the much-improved responsiveness and interactivity of the Linux desktop. While compiling the Linux kernel with 64 parallel jobs, 1080p video playback was still smooth, windows could be moved fluidly, and there was not nearly as much of a slowdown compared to when this patch was applied. Linus Torvalds has shared his thoughts on this patch: So I think this is firmly one of those 'real improvement' patches. Good job. Group scheduling goes from 'useful for some specific server loads' to 'that's a killer feature.'"
Re:Compiling the kernel (Score:5, Informative)
This isn't about improving performance of any one specific program; but about making a reasonably heavily-loaded system much more pleasant to use. Compiling the kernel is just a trivial way to generate a large amount of non-interactive CPU load and a bit of disk thrashing...
Re:Distros? (Score:5, Informative)
I exaggerate, but it's not far from the truth - the kernel releases are imported into the testing repository as soon as they come out.
Re:Wasn't there a desktop friendly scheduler rejec (Score:5, Informative)
They mention the "Con Kolvias" scheduler in TFA, but they don't seem to want to refer to it by its real name:
http://en.wikipedia.org/wiki/Brain_Fuck_Scheduler [wikipedia.org]
It doesn't scale well past 16 cores, which is why Linus doesn't want to include it in the main kernel. But it's included in custom kernels for mobile devices, such as CyanogenMOD for my Android phone.
Actually that sounbds quite large. (Score:4, Informative)
typo (Score:2, Informative)
shouldn't that read "and there was not nearly as much of a slowdown compared to when this patch wasn't applied"?
Re:Distros? (Score:3, Informative)
If you don't care about having it in a release, then the kernel team already has a PPA with all the releases + rcs + daily builds. I would think the next *buntu release (11.04) will have 2.6.37, so you probably won't see this in an official release until october next year...
Re:Distros? (Score:4, Informative)
Or whenever YOU choose to install the patched kernel. Freedom at work.
Re:This was always my biggest problem with Linux (Score:3, Informative)
Ubuntu.
And it'll probably be in Natty Narwhal, which is to be released in April 2011. https://wiki.ubuntu.com/KernelTeam/Specs/KernelNattyVersionAndFlavours [ubuntu.com]
Re:Wasn't there a desktop friendly scheduler rejec (Score:5, Informative)
The previous scheduler that Con wrote was rejected in favor of CFS which is currently in use by the kernel. CFS is at least partly based on ideas from Con, and he was also credited for them.
Re:Compiling the kernel (Score:3, Informative)
I know exactly what the problem is. Try passing different elevator options to the kernel at boot time. It helps with clock skew on virtual environments.
Re:Actually that sounbds quite large. (Score:3, Informative)
From what I gather (which isn't a lot, mind you), it's not so much optimizing the execution of an algorithm, but changing what the algorithm actually does. In this case: smarter scheduling.
Re:Compiling the kernel (Score:5, Informative)
Yep. For those that haven't tried it without the patch, a multithreaded kernel compile will typically peg a modern multicore CPU at 100% and will even give you drunken mouse syndrome. Just being able to scroll around a browser window while doing a lengthy make -j64 is impressive. Being able to watch 1080p video smoothly is ... astounding. Especially when you consider the minimum CPU requirement for 1080p H.264 playback is a 3 GHz single core or a 2 GHz dual core.
Re:Compiling the kernel (Score:2, Informative)
Or better: use a combined key of both tty and pgrp , so that we already see a benefit right now for the compilation example, and benefits later for all other cases (when window manager will start doing this, if they don't already)
This will still leave the "firefox with many facebook tabs open slows down webmail within firefox" case though, but maybe that case could be solved by taking a baseball bat to Mark Zuckerberg's head :-)
Re:will this help with the swap-paralysis problem? (Score:1, Informative)
512MB, wow! I wouldn't have guessed something like KDE could even be run on that little. I have 2 GB, and I see the above "endless swapping" problem about once every month or two running KDE 4.5. However, note that you can delay the time between seeing it if you restart KDE every week or so. It leaks memory over time, but restarting it "resets" its memory use, so if I remember to do that, I'll basically never see the swapping problem.
Re:Actually that sounbds quite large. (Score:5, Informative)
No, you went wrong by confusing "improving the decisions of the scheduler, so that the user experience is improved" with "making the scheduler run faster."
Re:teh snappy!!!! (Score:4, Informative)
Or rather, that there are different priorities.
The problem was with finer grained scheduling you wound up sacrificing server performance for desktop responsiveness. Linux does not want to sacrifice performance really.
To be fair I've been using linux on my desktop for over ten years now and even without this patch mostly fail to see an issue.
gentle linux restart (Score:1, Informative)
alt+sysreq REISUB
R: keyboard in raw mode
E: end signal to processes
I: kill signal
S: synch filesytems
U: emergency umount filesystems
B: reboot
Re:Distros? (Score:3, Informative)
What's cool is that Arch is actually pretty stable and works well, yet it's pretty bleeding edge (although to be honest this kernel will be in testing, not stable. probably in stable 2 weeks after 2.6.37 is released)
Re:Compiling the kernel (Score:4, Informative)
So no, it doesn't have to be a physical RS232 serial line like in the seventies :-)
Re:or, plan B: (Score:1, Informative)
Well for many netbook users upgrading ram isn't an option or it is a limited option. I would agree though that swapping to SSD isn't a great idea, really quick way to kill that SSD. though i don't know if swapping to NFS would be a real option on a netbook either where you are usually limited to wifi or 10/100 ethernet. I don't think many netbooks can go gigabit.
Not a big linux user here, but how does linux cope with running with NO swap space? I know it can be done on windows and once you hit that brick wall you just have to start closing down apps.
Unfortunately turning off swap makes it worse (Score:1, Informative)
See this bug which has made the last 3-4 releases of Ubuntu necessary to reboot daily like Windows ME:
https://bugs.launchpad.net/ubuntu/+source/linux-source-2.6.22/+bug/131094 [launchpad.net]
Re:Compiling the kernel (Score:5, Informative)
Re:Wasn't there a desktop friendly scheduler rejec (Score:5, Informative)
Its explained in the BFS FAQ http://ck.kolivas.org/patches/bfs/bfs-faq.txt [kolivas.org]
Why "Brain Fuck"?
Because it throws out everything about what we know is good about how to design a modern scheduler in scalability.
Because it's so ridiculously simple.
Because it performs so ridiculously well on what it's good at despite being that simple.
Because it's designed in such a way that mainline would never be interested in adopting it, which is how I like it.
Because it will make people sit up and take notice of where the problems are in the current design.
Because it throws out the philosophy that one scheduler fits all and shows that you can do a -lot- better with a scheduler designed for a particular purpose. I don't want to use a steamroller to crack nuts.
Because it actually means that more CPUs means better latencies.
Because I must be fucked in the head to be working on this again.
I'll think of some more becauses later.
Re:teh snappy!!!! (Score:5, Informative)
But the fact that Windows and OS X and many other lesser OS have fixed this problem years ago could also be used to show that Open Source is not quite as up on the times as you would like to think.
Seriously? OS X has absolutely terrible scheduling. A process that causes some swapping will cause beachballs everywhere and can freeze the windowserver for several seconds (well, the mouse keeps moving, but nothing else does). Hell, I have a FreeBSD VM on my Mac that responds better under load than stuff outside of it does.
Re:or, plan B: (Score:3, Informative)
I believe the parent is talking about the iowait bug #12309 [kernel.org], which, maddeningly, has nothing to do with swap, or filesystem type. You can turn off the swap entirely and still trigger the bug. Of course there are use cases where heavy swapping brings the system down, so there is a perceived improvement by most people when turning off the swap.
Re:will this help with the swap-paralysis problem? (Score:1, Informative)
> You do a *hard reboot* to recover? Why not just wait a few minutes, or better yet, kill X, firefox or any other process hogging an insane amount of memory and then wait a few minutes before restarting it.
Because the system becomes all but unresponsive. You can't interact with it enough to kill the processes - that might take half an hour or more. Moving the mouse to another window takes 10 minutes (literally). You can't use an alt-ctrl-F1 style text terminal either because the login times out before you can log in. There is no way to kill the process as you suggest.
> Don't blame Linux for when your system isn't up to spec.
I have two gigabytes in my laptop. I cannot upgrade it further.
If I was the only person with this problem I'd chalk it up to something odd about my system, but there are many others, even other Linux users I know locally who see the same issue. Also note that defects have been filed against this problem by lots of people. It is a real problem. Dismissing it does a disservice to the possible desktop acceptance of Linux.
Re:Compiling the kernel (Score:1, Informative)
If you meant to say 1200 bps modems
Well, if we're going to be really pedantic old-timers, we called it "1200 baud" in those days. ;)
Comment removed (Score:2, Informative)
Re:Wasn't there a desktop friendly scheduler rejec (Score:5, Informative)
BFS makes the classic trade offs which Linus and almost all others absolutely agree is a bad decision for core inclusion. BFS trades performance for latency. Basically you get really good interactive performance in exchange for massively losses in over all throughput and efficiency. Furthermore, BFS sits on a horribly slippery slope in that the level of *inefficiency* scales wonderfully with core count. Basically, the more cores you add, the more time BFS spends BF*cking around.
Basically BFS only makes sense on single or dual core systems which will primarily be used as a desktop. After that, it better be a dedicated desktop system because compared to the stock kernel, performance is going to royally suck, despite latency being fairly good. But then again, BFS was never really designed to address scalability - its always focused on latency and a superior interactive desktop experience. By most measures it works, but at a heavy cost.
Re:windows (Score:4, Informative)
Was I the only one thinking about Amigas and Alphastations?
Amigas had so much offloading that you could pull the CPU out and still move the mouse pointer.
Re:10GUI and similar GUIs are overrated (Score:2, Informative)
For instance, it'll be nice to be able to "alt-tab" from one browser tab to another browser tab (so much so that sometimes I open new browser windows just to be able to alt-tab between them).
Ctrl-Tab.
Re:Compiling the kernel (Score:4, Informative)
I also had to walk 10 miles, in the snow. Uphill both ways.
Re:Wasn't there a desktop friendly scheduler rejec (Score:3, Informative)
At first glance it really seems to improve system responsiveness in regards to user input. However this comes with a gigantic trade off. As an example: If I had a standard definition xvid file playing, the video would literally pause while the system loaded other programs like Firefox, Nautilus, etc. To me that is unacceptable. User input is nice, but if it's choking the entire rest of the system, it isn't something I want to use. The standard scheduler that comes with the kernel might not be perfect, but it will continue playing video while it opens up other apps.
Re:Compiling the kernel (Score:3, Informative)
To continue:
Baud != bps.
Baud is modulation changes per second, and in each modulation change there may be a representation of one or more bits which means that the modem may be 1200 baud but you got 9600 bps out of it due to the modulation (phase and amplitude of tone)
And the old Telebit Trailblazer modems with PEP protocol - they were fantastic in crappy conditions. Multi-carrier technology so that even if there were interference at least a few got through anyway and the only effect was that the bandwidth was quenched. Things to do datacom over barbed wire with! :)
Re:10GUI and similar GUIs are overrated (Score:3, Informative)
alt+tab deals with windows in terms of "most recently used".
ctrl+tab goes to the next tab in "position". ctrl+shift+tab goes to the previous tab in "position". They don't do "most recently used tab".
Re:teh snappy!!!! (Score:3, Informative)
That's pretty bad. On OS X, you used to be able to kill the system by mmap()ing a large file and touching one byte on each page in turn (I had a nice 10-line C program that would kill the system like this). With 10.5, it's a lot better. The system freezes for a few seconds but then it just becomes really slow, rather than dead.
OS X handles out of memory conditions a lot more gracefully than Linux. When Linux runs out of memory, it kills random processes until it can continue. When OS X almost runs out, it suspends processes and pops up a dialog telling you to close some. The resume button in this dialog doesn't work (yay Apple QA), but you can kill -CONT processes to resume them from the terminal. This is typically enough to kill the errant process that's decided that it needs 10GB of RAM and resume everything else. Basically, apps continue to work until they try to allocate more than a tiny bit more memory from the OS, then they're paused (unless they're on a small list of apps that are excluded from this because they're required to recover, for example the window server).
OS X doesn't do a terrible job, but to claim that they've solved this problem is a massive overstatement.
Re:This was always my biggest problem with Linux (Score:3, Informative)
If you cut back on the eye candy most distros have in their default desktop environments you'll wipe away any sluggishness. Switching to a simple window manager that doesn't use pixmaps for everything will significantly improve X's performance.
Re:teh snappy!!!! (Score:3, Informative)
(well, the mouse keeps moving, but nothing else does).
When that happens - MacOS, Linux or Windows, its usually because the mouse driver is interrupt driven and the mouse cursor is drawn by the video card. So the scheduler generally isn't even involved - the mouse generates an interrupt (pausing whatever task is currently scheduled on the cpu), the interrupt handler reads the mouse coordinates and hands them off to the video card, returning the cpu back to the regularly scheduled process and the video card redraws the location of the mouse on the screen.
Re:This was always my biggest problem with Linux (Score:3, Informative)
Windows is so annoying with the sidebar thing if you click a video and then try to delete it it won't let you delete it because it's caching the preview for the sidebar.
Re:or, plan B: (Score:5, Informative)
Actually even with no swap you will jam Linux when you run out of memory. Things like system libraries get thrown out of memory cache, but are soon needed again and read from the disk. This kind of circus can go on for half a hour until the actual OOM killer gets into the game.
Re:Actually understand the benchmark, then critici (Score:3, Informative)
You either missed it, or are conveniently "failing to mention it", but the idea was Linus' to begin with, and he wasn't expected such great results. There are reasons why Con's patches weren't used and this one is used. I'm not about to debate it here, as it has been beaten to death already, and with logic like "fuck Linus" you clearly have me outclassed in any such debate.
Re:Wasn't there a desktop friendly scheduler rejec (Score:5, Informative)
It's been fairly well documented but you still seem to ignore the reality of what happened.
http://kerneltrap.org/node/14008 [kerneltrap.org]
Read all that then tell me that Linus has an ego here. It seems to me that Linus is the only level headed guy and you're just trying to distort what really happened.
- He choose CFS over SD because SD behaviour was inconstant among users' computers
- Con would argue with people sending him problems rather then accept them as problems with his code
- Linus didn't want code in the kernel that would only work well for certain users
- Linus didn't want code maintained by someone that was so hostile to others' critique
- Linus states that he believes the desktop is an important platform