Significant Interactivity Boost in Linux Kernel 673
An anonymous reader writes "The Linux kernel team is at it again. Linux creator Linus Torvalds recently proposed a patch to offer interactive processes a boost, greatly benefiting the X desktop, as well as music and movie players. O(1) scheduler author Ingo Molnar merged Linus' patch into his own interactivity efforts, the end result nothing short of amazing... The upcoming 2.6 kernel is looking to be a desktop user's dream come true."
FINALLY! Thank you! (Score:-1, Insightful)
I cannot tell you how long I've been waiting for something like this. As an avid Microsoft fan, one of my biggest beefs was the inferior performance of the Linux GUI and its components. Although I will learn the console eventually, I will still probably use the GUI as my cheif method of computing since I find it very fast and efficient. Maybee this will finally blur the line between OS's enough to get more people to switch over.
Re:left, no right! (Score:5, Insightful)
Re:left, no right! (Score:3, Insightful)
Re:left, no right! (Score:1, Insightful)
Re:FINALLY! Thank you! (Score:2, Insightful)
"As an avid Microsoft fan..."
And you admit this on Slashdot?! You are brave.
The challenge of scheduling for interactivity (Score:5, Insightful)
Note that responsiveness is not something that will scale with increasing processor speed. So these patches could really have an effect in setting the Linux desktop experience apart. Will this help to capture desktop market share? Whoda thunk it :)
Re:left, no right! (Score:5, Insightful)
How about this radical idea--
Let Red Hat, SuSE, etc. compile different kernels with different options and install them as needed
Err (Score:5, Insightful)
That would depend on exactly what you talking about. Those linux users running something like Blackbox would laugh at you for saying so. I'd also suggest as a user of both, KDE and XP have about the same interactive performance as well.
There's no doubt Windows still has more polish than Linux as a whole when it comes to the desktop. And while anything that improves any of LInux's many "gui's" is a welcome event, Linux's gui's are hardly inferior performance-wise across the board like your implying.
"Maybee this will finally blur the line between OS's enough to get more people to switch over."
Performance doesn't rate very high on why windows users aren't switching over. Lack of familiar apps and games, lack of widespread OEM bundling, and lack of millions in marketing are what's keeping people from switching over.
Re:left, no right! (Score:4, Insightful)
Re:X11 Beh. (Score:1, Insightful)
Re:left, no right! (Score:3, Insightful)
The better answer is to either a) make this option compile time, as someone mentioned, or b) make this option configurable (a la sysctl) at runtime. This would allow distribution maintainers to adjust the setting to match the type of installation they are doing, and users on stock installations to quickly adjust the kind of scheduling they have, just like the little check boxes in windows NT/XP.
Nice that Linux finally caught up with BSD... (Score:3, Insightful)
Anyway, I hope Linus and Ingo now have some spare time for real dreams of power users with Linux desktops - drivers! I used to run Linux at work and at home (used to, because I got rid of the Intel box at home). Between these two machines, I had an NVIDIA card, lucent WinModem, CLIE, Zaurus and an NTFS partition that wasn't recognized by default Redhat kernel.
After every kernel upgrade, I had to recompile 5 drivers. CLIE and Zaurus drivers came in the form of patches that usually refused to apply, or caused a hang when the device was attached! Once I tried a 2.5 kernel because it had some features, like suspend and resume that I could really use. While the default configuration built Ok, once I enabled the drivers I wanted, I couldn't get the thing compiled even before applying my patches.
Yes, you could just run "Redhat operating system", never upgrade the kernel and wait a few months to install a new release. Then you might find binary drivers to download for the kernel for your particular kind and number of CPUs. But the whole point of Linux (on a personal desktop) is to have some fun and try new stuff out easily.
Linux developers really need to stabalize driver interfaces. I should be able to go to kernel.org and download the latest kernel *binary*, then install a binary driver from the CD-ROM that came from my NVIDIA card.
USB and Firewire buses should be exposed by kernel as network interfaces, accessible to user programs through socket API. In this way, USB drivers will be both easier to write/debug AND will not contribute to Oops. For the ultimate of cool, Wine should support Windows USB drivers (my Virtual PC does!) and I should be able to just install Palm and Zaurus desktops and use them rather trying to feed ttyUSBNN to kpilot.
Having a stable system that doesn't have to be rebooted to Windows to use some unsupported USB device is far more important than raw performance. I wish any system's developers - Linux, *BSD, Darwin, BeOS, etc - would concentrate on this goal before going back to play with cool toys.
Re:FINALLY! Thank you! (Score:5, Insightful)
Its not just a UI issues; it does relate to the kernel in that the kernels job is to manipulate process priorities and give CPU cycles where they "should best be given". This is actually best done at the kernel level, and NOT the GUI level, because the GUI does not know about the other non-GUI processes is "competing with". I've felt for a long time that something like this should be done in both Windows AND Linux.
Windows is TERRIBLE at this. Consider the following scenario, which most here who here run Windows XP will be able to identify with. You boot up, you've just logged in. The task bar is there on the screen, the start button there, you click on it. And nothing happens. You wait. Still nothing happens. You wait some more. You start to get annoyed and click the start button a few more times. The hard disk is grinding away while Windows XP does all sorts of "invisible stuff" in the background. The computer is about as responsive as a brick. Then after anything from 20 seconds to a minute, the start menu suddenly opens and closes rapidly in quick succession a half dozen times.
THIS IS NOT HOW COMPUTERS SHOULD BEHAVE. Its pathetic. This is a perfect example of the necessity of this. The task bar process doesn't know about all those other background processes hogging CPU after you log in; there is no elegant way for it to magically know when to set its priority temporarily high, and for how long. But the kernel can say, OK, the user is trying to press a button, we must respond, and temporarily boost the start bar (explorer.exe) process and block the others.
On desktop machines (i.e. not servers), user input is the most important thing. If the user presses a button, something must happen. The kernel should be continually shifting priorities around to where the user is focusing his/her input.
Re:FINALLY! Thank you! (Score:3, Insightful)
But the kernel can say, OK, the user is trying to press a button
Just to preempt those people who are about to jump down my throat because "the kernel is not supposed to know about things like buttons", I know that, but thats not what I meant. I was speaking on a more abstract / higher level, but obviously this can still be implemented in terms of lower down OS things, e.g. the Win32 message queue and HWND system: the "OS" *does* know when, for example, when mouse click messages are posted to the DefWndProc of an HWND, and it does know which process is associated with that HWND, etc. In the Linux OS design view, this isn't part of the kernel, no. But in Windows, this is just one layer above what Linux people would classify as being "the kernel"; in Windows there is a lower degree of separation between the two.
Re:X11 Beh. (Score:5, Insightful)
But even considering the larger and more featured system, X11 is as fast, or faster, than windows on all but one of my machines, the one where its slower is because the graphics card is very poorly supported by X.
What causes slowness more than X11 itself, is the programs running on top of it... KDE for instance, its hardly a speed demon compared to say, windowmaker.
You dont need a high end videocard to make X smoothe, you just need one that`s well supported... my PCI ATI Rage Pro works perfectly, as does my Elsa GLoria Synergy, both are oldish 8mb pci cards.
Any system will completely suck with poor drivers, try configuring windows to use generic vga or vesa drivers if you want a laugh.
X11 is FAR superior to any local-display-only gui system, i have several machines here, and 1 monitor for X, apps running from each machine and displayed here and interacting smoothly with each other and with locally running apps.
Re:Simply More Evidence (Score:2, Insightful)
Why, yes.
If I have the code, it's more valuable to me.
Re:NT (Score:3, Insightful)
Re:Doesn't work for me (Score:3, Insightful)
Personally i very much like the way X11 and the linux console handles cut+paste, its perfect for me, fast and doesnt require me to keep jumping between the mouse and keyboard. Ofcourse it would be better if it was configureable to satisfy people such as yourself, and also for machines with 1 button mouse.. i have to use an old mac sometimes.
Real-Time OS not something you'd want... (Score:3, Insightful)
A lot more important work is to ensure "fairness", so that no processes is very slow, and in particular that near-real time processes (like playing a movie, 1/30th of a second is not real-time, it could render it _way_ faster) get "enough" time to make it play smoothly.
From what I've been able to gather, both have been greatly improved in 2.6...
Kjella
Yes, thank you (Score:3, Insightful)
Go to xterm, try to unzip a 1 gig zip file (on a HD on that box) and the open mozilla and drag the window around...
Wait wait wait, mouse quits moving... Then it starts jumping all over the screen. Time for a coffee.
This is part pager and part interactive task/busy background task thing that these patches try to fix.
That was a big turn off of the 2.4 kernel for me.
m
Re:FINALLY! Thank you! (Score:3, Insightful)
Another thing: try this on Windows 98. While Windows 98 is booting up, just as the desktop gets drawn, press the windows key (or wait till the task bar is just shown).
Windows 98 crashes - you have to ctrl-alt-del and select shutdown.
Re:Why not a real-time scheduler? (Score:5, Insightful)
X is the most important example, since it doesn't help how much CPU your xterms or other X clients you run get if X doesn't get enough CPU time to service them, as if X doesn't get enough time the only thing extra CPU time will give your x clients is the ability to go back to sleep faster and more often.
Realtime scheduling is something else alltogether. Realtime scheduling is about predictability, not about CPU time allocated. With a realtime scheduler you can guarantee that task A get some time at least every 10ms, for instance, but if you're maxing out the CPU you still need a way of deciding which tasks have priority, or reduce their overall time slices.
The kernel patches in question attempts to decide which tasks to give priority automatically, instead of resorting to hacks like using nice on specific processes. It achieves that by making the assumption that if task A is interactive, and it frequently waits on B, then task B needs to get more CPU too.
Since a high load desktop scenario will likely have lots of clients waiting on X the result is that X will get more CPU even if X itself isn't an interactive task, and hence the machine will hopefully feel more responsive.
The way this is being accomplished is good because it doesn't special case - any non-interactive task that provides vital services to interactive tasks will get more CPU (though in this particular implementation, I believe only if they communicate using Unix domain sockets), without the user or developers having to guess which processes should get it.
Re:left, no right! (Score:3, Insightful)
Users are dumb.
Not necessarily the users using the system, but the users developing software for it. If you give them the option of choosing whether their program is scheduled as an interactive process or a batch they will always choose the wrong one.
"Why yes, that is my elitist attitude you are observing. Please be careful with... Doh!"
-Adam
Re:FINALLY! Thank you! (Score:1, Insightful)
Re:Err (Score:2, Insightful)
Re:left, no right! (Score:2, Insightful)
Plenty of people, particularly java developers, like to run local instances of various servers on the same box they use as a workstation. This can often increase a developer's productivity in certain development environments for small-to-medium sized projects.
Don't think every user is exactly like you and has no need for the proposed change. Proposals such as this, and their subsequent implementation, are the primary strength of open source.
Re:explanation needed, please (Score:3, Insightful)
Old fashioned managers handled this by simply giving you an outline of the window you were moving and then doing a full screen redraw. The newer system (say the last 10 years) has been to actually do this real time with dozens of small redraws. This is very CPU intesnive. Further there is an accurate measuring device (the human eye) which notices lacks of performance very quickly.
On a single tasking machine where the computer could dedicate itself to doing this task this still wouldn't present a problem. On a modern OS with dozens of processes running doing things like the above is a scheduling nightmare. A very complicated scheduling system is out because the scheduling system needs to run so many times a second. The result is you need to think of cool tricks to make this work.
The cool trick that windows thought of was using the notion of "forground" and "background" to and thus giving certain apps a massive CPU advantage while others got little if any attention except when the CPU wasn't busy. The problem with that is that it forces the user to either:
a) Have extremely unreliable background processes (default for windows NT/2000/XP desktop)
b) Have an unresponsibe desktop (default for the NT/2000.. server)
More importanly this forground/background solution doesn't solve the problem of competing demons which are having other types of CPU problems.
So Linux long ago rejected this cool trick and there is no notion of "forground" and "background". They were going to handle the problem right or not at all. Now with tuning you can make this particular X related problem go away but only at the cost of introducing other problems, and more importanly all you are really doing is covering the symptom of the disease not curing the diesase. The kernel group has figured out a general solution which improves scheduling for all applications without substantially increasing the complexity of the scheduler.
Unlike many kernel improvements this one will have substantial impact on desktop performance right out of the box when distributions start shipping with the 2.6 series which is why people are excited.
Re:left, no right! (Score:2, Insightful)
Re:Simply More Evidence (Score:1, Insightful)
Please direct any assertions that Solaris is "Open" to
Re:Copy & Paste behavior is the BEST thing abo (Score:3, Insightful)
Re:FINALLY! Thank you! (Score:3, Insightful)
Completely wrong. Most people only care that their computers work reliably for up to 8 hours at a time, and shut them off when they're not in use.
Most people don't 24x7 uptime, and wouldn't want it anyway: computers use quite a bit of power, and power costs money.
Indeed, most people I know turn their computers off when not in use.
Re:Linus discovers priority inversions - WRONG (Score:5, Insightful)
Which is wrong. Did you read the article?
Priority inversion is, as you explained yourself, about a high priority task effectively getting a low priority by being dependend and therefore waiting on a low priority task.
The article is about tasks at the same priority[1]. The task scheduler distinguishes between interactive and non-interactive tasks in order to improve latency where the user cares.
Beforehand, the behaviour failed on a slow, loaded system to recognize the X server as interactive, because then X looks like a CPU-hog[2]. That resulted in freezes of several seconds[3]. Simply speaking, the patch solves this by passing some interactivity points between processes.
You could have easily seen that this is not about priority inversion as one of the suggested work-arounds was to simply increase the niceness of the X process (which wouldn't help, if priority inversion had been the problem).
Regarding QNX: As good as it is as a RTOS, as bad it fails to do something sensible when you have too much processes at the same priority. Having a reasonably working system presumes that each task is assigned an appropriate priority. Of course, the people at QNX did a decent job on the default priorities.
[1] It may have an impact on tasks of different priority. I did not care to investigate that aspect, because that is of minor importance to what the patch is about.
[2] And for the scheduler, interactivity is determined by a process going to sleep often (by waiting for interaction).
[3] For non-interactive processes it is beneficial to do them in larger hunks, i.e. let 5 seconds other processes do their work, then work 5 seconds, instead of having 0.01 second slices and do the switching all the time.
Re:Linus discovers priority inversions - WRONG (Score:3, Insightful)
When you treat tasks differently, you're prioritizing them. All the priority information isn't necessarily encoded into the UNIX-type priority number. This is a nomenclature distinction between "priority" in the formal sense of "who gets the (a) CPU", vs. the classical UNIX representation of priority numbers.
This is the classic X argument (Score:4, Insightful)
Ugh. I don't buy it.
To put it in perspective, lots of Unix has a big organization problem. X is just emblematic. It's "lower-level" APIs are a big stinking mess. Ever tried to program against it without a super-high-level bit of middleware? Then let's talk about how nice it is. If you're not up on this, try reading JWZ's rants on it (many written as he was porting Netscape)? X is a 4 foot high sandwich of crap, layer after layer between you and the display, full of massive, sucking complexity, the bugs, inefficiency... even during this supposedly wonderful "network transparent" windowing this foul stew shows its colors, as no combination of two applications or X servers quite looks the same. It's a verifiability nightmare, too, of course (and for instance, disabling X's many attempts to listen and talk on the network are one of the first things you do to secure a machine properly - and for real security, you avoid installing X altogether).
The API design itself is atrocious. The much-touted "flexibility" is really code for laziness - it was a lot of work to do a proper GUI, so no one did it. The mishmash of X server extensions, window managers, font handling systems, etc. that's been cobbled together has led to a nightmare for both programers and users, as any given application doesn't just require "X", but a complex recipe of libraries and versions, and an end-user experience where no two applications look or act the same... or even remotely similar... Where cutting and pasting between windows is a pipe dream, and young geniuses still struggle to configure fonts properly for linux distributors.
Or to just put it plainly, as my friend (who from time to time would write X windows gadgets) would say, it's only about twice as hard as managing the video memory yourself.
"And thank god it's not all standardized, or we'd never have had all those wonderful experiments with different ways to do a GUI that never actually happened." In practice, no system is immune from its initial design choices, and it's been an endless series of awful MacOS knockoffs, multi-button madness, color-pallete spinning goofiness. Is X11 a "GUI experimenters toolbench?" Then I think it's time for something a little more grounded in everyday realities of computer use.
I'm not even warmed up yet. I mean, X is still peppering the filesystem with a hedge-maze of exotically formatted text files describing the hex colors of every pixel of the trim of every window for a variety of appliations and classes in a complex inheritance and assignment scheme that few X developers even understand. Check it out, your XDefaults are "human readable."
Shall we even discuss its security model?
Modern Linux has tried to make its peace with X through wrappers, and we write against Tcl/Tk, Qt, inside the Gnome or KDE framework, and yet still the focus groups come back crying... we try to blame overfamiliarity with windows, but the problems are bigger... all of Unix (and of course Linux) suffers from the same class of problems that X does; as, for instance, an application needs to prompt you to insert a series of CD's, but there is no "single, authoritiative, standard" place to go find out what CD drives are installed on the computer, and what their device names are (yes, we know what they _usually_ are), and finding out if any of the CDs are already inserted involves parsing the text output of a proc file or a mount command, and so on and so forth... And all of this is being done by a messy bash script... so it's no surprise this functionatlity is broken even in, for instance, RedHat's own v8 package manager... I hope you can grasp the metaphor.
It's a mess. Patches won't clean it up. Frankly, it's time we took the whole GUI back to the drawing board. But even if MacOS is the end-all/be-all, we can do it a hell of a lot better than we do in X.
Following are some choice quotes from Don Hopkins' [art.net] essay:
X-Windows is the Iran-Contra of graphical user interfaces: a tragedy of political compromises, entangled alliances, marketing hype, and just plain greed. X-Windows is to memory as Ronald Reagan was to money. Years of "Voodoo Ergonomics" have resulted in an unprecedented memory deficit of gargantuan proportions. Divisive dependencies, distributed deadlocks, and partisan protocols have tightened gridlocks, aggravated race conditions, and promulgated double standards.
X has had its share of $5,000 toilet seats -- like Sun's Open Look clock tool, which gobbles up 1.4 megabytes of real memory! If you sacrificed all the RAM from 22 Commodore 64s to clock tool, it still wouldn't have enough to tell you the time. Even the vanilla X11R4 "xclock" utility consumed 656K to run. And X's memory usage is increasing.
...
X was designed to run three programs: xterm, xload, and xclock. (The idea of a window manager was added as an afterthought, and it shows.) For the first few years of its development at MIT, these were, in fact, the only programs that ran under the window system. Notice that none of these program have any semblance of a graphical user interface (except xclock), only one of these programs implements anything in the way of cut-and-paste (and then, only a single data type is supported), and none of them requires a particularly sophisticated approach to color management. Is it any wonder, then, that these are all areas in which modern X falls down?
...
As a result, one of the most amazing pieces of literature to come out of the X Consortium is the "Inter Client Communication Conventions Manual," more fondly known as the "ICCCM", "Ice Cubed," or "I39L" (short for "I, 39 letters, L"). It describes protocols that X clients ust use to communicate with each other via the X server, including diverse topics like window management, selections, keyboard and colormap focus, and session management. In short, it tries to cover everything the X designers forgot and tries to fix everything they got wrong. But it was too late -- by the time ICCCM was published, people were already writing window managers and toolkits, so each new version of the ICCCM was forced to bend over backwards to be backward compatible with the mistakes of the past.
The ICCCM is unbelievably dense, it must be followed to the last letter, and it still doesn't work. ICCCM compliance is one of the most complex ordeals of implementing X toolkits, window managers, and even simple applications. It's so difficult, that many of the benefits just aren't worth the hassle of compliance. And when one program doesn't comply, it screws up other programs. This is the reason cut-and-paste never works properly with X (unless you are cutting and pasting straight ASCII text), drag-and-drop locks up the system, colormaps flash wildly and are never installed at the right time, keyboard focus lags behind the cursor, keys go to the wrong window, and deleting a popup window can quit the whole application. If you want to write an interoperable ICCCM compliant application, you have to crossbar test it with every other application, and with all possible window managers, and then plead with the vendors to fix their problems in the next release.
In summary, ICCCM is a technological disaster: a toxic waste dump of broken protocols, backward compatibility nightmares, complex nonsolutions to obsolete nonproblems, a twisted mass of scabs and scar tissue intended to cover up the moral and intellectual depravity of the industry's standard naked emperor.
Using these toolkits is like trying to make a bookshelf out of mashed potatoes.
- Jamie Zawinski
...
The fundamental problem with X's notion of client/server is that the proper division of labor between the client and the server can only be decided on an application-by-application basis. Some applications (like a flight simulator) require that all mouse movement be sent to the application. Others need only mouse clicks. Still others need a sophisticated combination of the two, depending on the program's state or the region of the screen where the mouse happens to be. Some programs need to update meters or widgets on the screen every second. Other programs just want to display clocks; the server could just as well do the updating, provided that there was some way to tell it to do so.
...
.Xdefaults", but if he happens to have copied Fred's .xsession, he does an xrdb .xresources, so .Xdefaults never gets read. Susie either doesn't xrdb, or was told by someone once to xrdb .Xdefaults. She wonders why when she edits .Xdefaults, the changes don't happen until she 'logs out', since she never reran xrdb to reload the resources. Oh, and when she uses the NCD from home, things act `different', and she doesn't know why. "It's just different sometimes."
.xresources (or was it a file that was #included in .xresources) of the form *fucked*fontList: 10x22, which he copied from Steve who quit last year, and that of course that resources is 'more specific' than his, whatever the fuck that means, so it takes precedence. Sorry, guy. He can't even remember what application that resource was supposed to change anymore. Too bad.
What this means is that the smarter-than-the-average-bear user who actually managed to figure out that
snot.fucked.stupid.widget.fontList: micro
is the resource to change the font in his snot application, could be unable to figure out where to put it. Suzie sitting in the next cubicle will tell him, "just put it in your
Joe Smartass has figured out that XAPPLRESDIR is the way to go, as it allows him to have separate files for each application. But he doesn't know what the class name for this thing is. He knows his copy of the executable is called snot, but when he adds a file Snot or XSnot or Xsnot, nothing happens. He has a man page which forgot to mention the application class name, and always describes resources starting with '*', which is no help. He asks Gardner, who fires up emacs on the executable, and searches for (case insensitve) snot, and finds a few SNot strings, and suggests that. It works, hooray. He figures he can even use SNot*fontList: micro to change all the fonts in the application, but finds that a few widgets don't get that font for some reason. Someone points out that he has a line in his
...
On the whole, X extensions are a failure. The notable exception that proves the rule is the Shaped Window extension, which was specifically designed to implement round clocks and eyeballs. But most application writers just don't bother using proprietarty extensions like Display PostScript, because X terminals and MIT servers don't support them. Many find it too much of a hassle to use more ubiquitous extensions like shared memory, double buffering, or splines: they still don't work in many cases, so you have to be prepared to do without them. If you really don't need the extension, then why complicate your code with the special cases? And most applications that do use extensions just assume they're supported and bomb if they're not.
Cool, they're reinventing Windows (Score:2, Insightful)
Heh, nice to see them giving up and implementing the priority boost. I wish them the best luck implementing the detection of "foreground" tasks, with an user interface with so little contact with the processes it serves
Don't you know what this patch is about? Well, ever noticed how on Windows the three/four most used programs among the currently open tend to display their windows instantly when switching between them, not cause the disk to swap at all, be generally more responsive? It's because the Win32 subsystem gives foreground tasks a slight sheduling priority boost, and frees up background tasks' unused resources (the on-screen buffers of windows, I guess) as required by the foreground tasks' needs
You (and I mean you, random Slashdotter talking out of your ass) can easily see how X11 can't possibly compete on equivalent hardware, no matter how hard they try: