Significant Interactivity Boost in Linux Kernel 673
An anonymous reader writes "The Linux kernel team is at it again. Linux creator Linus Torvalds recently proposed a patch to offer interactive processes a boost, greatly benefiting the X desktop, as well as music and movie players. O(1) scheduler author Ingo Molnar merged Linus' patch into his own interactivity efforts, the end result nothing short of amazing... The upcoming 2.6 kernel is looking to be a desktop user's dream come true."
Re:Creepy... (Score:1, Informative)
Actually... (Score:4, Informative)
There was a time .... (Score:5, Informative)
Linux still screams, I have a single server with two gig's of ram in it that runs 100 desktops (KDE) simultaneously. Yes it indeed takes alot of ram to run all of the new software. But for a machine that runs 2200 processes that is a impressive feat. It is a dual processor box and I have yet to see it reach over 30% processor utilization, a testament to the efficency of the kernel.
Software today requires a ton of ram, this has nothing to do with efficency of the linux kernel.
Along with this goes the idiots that think there is something wrong with X. I run this stuff in a corporate environment and X windows is linux's biggest strength. Remove X Windows and I would have to eliminate our corporate use of Linux.
Re:explanation needed, please (Score:5, Informative)
The kernel development team are experimenting with heuristics to determine what processes are "interactive" and to determine "how interactive" those processes are.
An interactive process is a process which spends a portion of it's time sleeping, waiting for some kind of event, and then needs cpu time quickly after the event happens.
In this case the events are user input and screen redraw requests.
So, the trick is that interactive processes don't need any more CPU time than other processes, they just need it very quickly in response to requests. Low latency.
The question is, how do you determine what processare are interactive, and how interactive they are.
They have developed a system whereby there are effectively "interactivity points" that can be given to and taken away from a process.
The act of being woken up from sleeping by an event awards you interactivity points. The act of completely using up lots of timeslices (acting like a CPU-bound process) takes away interactivity points.
With Linus's new patch, once you've reached a certain threshold of interactivity points, some of your points start going to the process that woke you up. So, if an "interactive" process is always waking up in response to an event from a certain other process, than that other process is also awarded interactivity points.
In the end, your interactivity points are taken into account when choosing which processes get the CPU.
So, with this new code, processes which are "interactive" like your X11 apps get more of the cycles they need when they need them, decreasing their latency, and making them appear to work "better."
Justin Dubs
NT (Score:2, Informative)
The "feature" of biasing the scheduler either towards interactive proceses or to background processes has been around since NT 3.51, if I remember correctly. It was definitely in NT4, released in 1994 (again, IIRC). So, while this is welcome, it's not an innovation, and saying that Linus "proposed" it is misleading.
Re:Simply More Evidence (Score:4, Informative)
AmigaOS and windows are both fairly similar in purpose and features tho, unix is more tailored to heavy duty server use, and thin clients, and ofcourse its far more powerfull and flexible. Thus you have a powerfull stable kernel, multiuser abilities, and features such as remote displays and authorization in X.
True, windows has tried to copy some of the age old unix features, but the basic design remains the same with extra things kludged in as an afterthought, and theres still no X style remote apps managed by your local wm, its whole desktop or nothing.
So while windows may be faster on a single machine, due to its simpler design, once you scale up.. to say one server serving hundreds of thin clients, unix really pulls into the lead.
Re:Amazing! (Score:5, Informative)
Re:NT (Score:4, Informative)
Patch explanation (Score:3, Informative)
But the good thing about the patch is that it will also allow other servers (HTTP, SMTP - not only the X11 server) to be equally interactive and more responsive to external input.
Re:FINALLY! Thank you! (Score:4, Informative)
I have to say that KDE 3.1 is pretty snappy on my measly PII 400 with 320 MB of RAM under Gentoo Linux.
Saying KDE is slow is fudding.
Re:Simply More Evidence (Score:3, Informative)
Now that you say it, I also realize that no Windows machine that I have been using has ever been as responsive as the old Amiga. Of course this is also a hardware issue: the Amiga had pretty strict timing for all I/O operations and memory access. The different subsystems had their own time slots in DMA, which was based on the video refresh timing. To a certain amount other subsystems, like the "Blitter", could steal DMA cycles from the CPU. There was even a chip, the "Copper", that could perform certain actions based on the position of the electron beam of the monitor. In my opinion, this chip was the key to most of the impressive effects that could be produced by the Amiga. OTOH, such a design is pretty hard to scale w.r.t. speed. In the end, the Amiga declined because Commodore neglected hardware development for too long.
Re:left, no right! (Score:5, Informative)
Check out the Solaris 9 Resource Manager [sun.com], which can do both types. It allows you specify at a high level how much of the system's resources each group of processes gets under which conditions. You could say for example, group A (interactive) gets up to 100% unless group B (batch) needs some, in which case allow B up to 30% during the day and up to 70% at night. You could do this sort of thing in VMS over a decade ago. Also, even if the underlying OS doesn't give you the capability, an Oracle server running batch and interactive tasks can do it too.
First hand experience (Score:2, Informative)
This is wonderful. Kudo's and a round of whatever ya drink to the kernel crew.
Re:Actually... (Score:1, Informative)
Re:left, no right! (Score:5, Informative)
Suppose there are two CPU-bound processes marked as 'batch' and one process marked as 'interactive' which spends most of its time waiting for user input, but needs to respond quickly in short bursts when that input happens. The interactive process will get high priority and preempt the two batch processes when it needs to run; but when it goes back to sleep the two batch processes are scheduled with long timeslices.
Re:Actually... (Score:2, Informative)
Now it would be nice If we could run the X Server code on a seperate hardware, the gfx card. Unfortunately the X Server is too complicated to run on today's gfx cards. But the trend in gfx cards is clearly towards more independent computing power on the cards. Therefore it would be unwise to remove a layer of abstraction that we probably could use in forseeable future.
Re:Doesn't work for me (Score:3, Informative)
man xclipboard.
It has been part of the standard X11 distribution (thus, also XFree86) for ages.
Re:Actually... (Score:3, Informative)
Re:Actually... (Score:4, Informative)
And the real reason apple didn't go with X is because they wanted to use the OpenStep API and that's written to use a display postscript backend. It was easier to change those slightly to use a similar display pdf backend then it would be to rewrite them to use the completely different architecture of X (X is missing things like vector manipulations, resolution intependant objects and generally everything display postscript/pdf does well).
Re:Is this really a good thing ? (Score:2, Informative)
Before, the scheduler put processes that hogged the CPU in batch mode. They got bigger chunks of the CPU time but they had to wait longer for it as well.
With this patch if a lot of processes are waiting for a different process then that process should get CPU time with out waiting as long. On the other hand, if the process ends up taking a long time, it gets put into batch mode.
So a lot if a lot of windows are waiting for a response from X then X gets the CPU without waiting. Since X doesn't hog the CPU for a long time it doesn't get put into batch mode.
On the other hand, when you compile the kernel make is waiting for gcc to finish, but gcc uses a lot of CPU time so it gets put into batch mode.
It's a pretty clever idea. It speeds up playing mp3s at the command line instead of just trying to boots whatever is at the front of the screen (a la Windows).
Re:for all you... (Score:3, Informative)
Gentoo, while a great idea, isn't _that_ much faster than other distributions once this fact is taken into account.
Remember, 20% of the code is run 80% of the time, and you get your big performance increase by optimising that.
Re:Actually... (Score:5, Informative)
It doesn't. It goes through a UNIX socket. There is a significant difference.
I somehow doubt there was a single reason.
Concentrating on the UNIX socket is a mistake anyway. You need some form of client/server separation; otherwise you could never run more than one client. You also need some form of synchronisation between the clients and the server; otherwise you would have two clients accessing the hardware at the same time and most video hardware would simply lockup. The synchronisation method could be locks or mutexes or message passing or sockets; X11 chose a socket because that gives you UNIX sockets (local, high speed) and TCP/IP sockets (remote, flexible) without needing to code for special cases. Network transparency "for free".
Now the real question with X11 is "who should control the hardware". With X11 they decided a single process - the server - should control the hardware. This is perhaps the serious argument against X11. There are several reasons why this hurts performance but the serious problem - the one you inelegantly complain about - is that the client has to bundle all drawing requests up and send them to the server.
But stop. What's the real problem here. It's not that the bundling had to occur. No matter what model you chose there would have to be some data bundled up and sent between client and server. The real problem is the quantity of data. In Windows the quantity is a single message which is always quite small. In traditional X11 the "message" (aka request) grows without bound. If you're passing a huge bitmap then the request will be several kilobytes. Network transparency comes at a cost.
But stop again! Is this really a problem? The answer is no. X11 is extensible. All of the problem cases - bitmaps, video, 3D - can be special-cased with extensions. So on XFree86 we have Xvideo, MITSHM and DRI. In a traditional X11 model these guys would have stuffed the pipe to overflow and everything would have gone to shit. In modern XFree86 there is a second path that bypasses the pipe. You'll notice that DRI even allows the client to directly access the hardware! Network transparency is still there but can be bypassed on a needs basis. Perfect.
Now your argument shouldn't be "why do we need a client/server model" but "could we use something faster than sockets". The answer is no. There has already been work done by the XFree86 team where they tried a shm transport. It's no faster. Linux sockets are simply too quick. There's no reason to think that message passing would be any faster: effectively the X11 socket is a highly tuned message passing API. The platform independent nature of X11 means you'd need to use a platform independent message passing API. That probably means RPC or CORBA; X11 is going to be faster than either of those.
Anyway, my point from all of this is that the performance problems you complain about are being fixed. The developers are not idle and they are not stupid (far from it). If you wanted somebody to make your desktop faster then you could do no better than to put your trust in the current XFree86 developers team. They are a truly remarkable group of developers. They are not ignoring the performance problems. Give them some credit for understanding the depth of the issues rather than the superficial "why does XFree86 use TCP/IP?" misunderstanding that tries to pass for constructive criticism.
Re:explanation needed, please (Score:3, Informative)
How to fix this: (Score:3, Informative)
Not quite the same (Score:4, Informative)
Now, *that*, I have to say, is bordering on the flamebaitish -- yes, I see what you're trying to say, but that's a kind of offensive way to phrase it.
Uh...check out Windows 2000 scheduling algos.
It's not exactly the same thing, though even Linus mentioned it (rather offensively, IMHO, to Ingo). Windows has a simple heavy priority boost it gives to the foreground app. That works fine if you're working in a fairly modal manner on a single-user system and you have a desktop-with-foreground-and-background paradigm as a fundamental part of the OS.
Linux's scheduler takes a somewhat more ambitious (granted, that probably means you can trick it more nastily) approach, partly because it has a more general, more difficult task. From what I can tell from skimming the conversation, Ingo's work is something more along the lines of advancing the traditional UNIX approach of "this app didn't use (or is tending not to use) its full timeslice, maybe because it's blocking on I/O, so give it higher priority to get another timeslice than an app that *did* use its full timeslice". He's just doing somewhat more sophisticated automatic classification of whether an app is "interactive" or not.
Yes, on the very surface, it's similar in goal. Make the task that the user is working with get more cycles at appropriate times to reduce latency of interface response. However, the approach is very much different, and the potential benefits are higher (since this automatically addresses a wide range of apps, not just making the foreground app peppier to keep scrolling snappy).
I *will* give you that this has little to do with open source. I suspect that there are plenty of closed-source systems that have tried to do more advanced classification of apps as interactive or noninteractive.
Linus Patch works in 2.4.x (Score:3, Informative)
The interactivity still wasn't perfect, but it was noticably better. Now if I can just track down and apply Ingo's patch as well....