Intel Chief Evangelist Comments on Linux Scheduler 178
ByeByeWintel writes "James Reinders is Intel's Chief Evangelist for Intel's Software Development Products. In a
recent interview on Devx.com he stated: 'If I could get ONE wish fulfilled would be for OS scheduling to focus on processes, and not threads, for scheduling. And demand that processes manage their scheduling of threads ... There is a lot of opportunity for operating systems to offer these types of control in the 'running of applications' interfaces. I'd like an OS to let me specify the 'world' my application runs in (which processors, how many, etc.) These interfaces are available in Windows at run time (the task manager will let you adjust where a running task can go).'"
I don't understand (Score:5, Informative)
Re:Hello?? (Score:3, Informative)
Re:So do it (Score:5, Informative)
Re:which processors (Score:3, Informative)
Yes, and has for at least a decade. As has SGI's IRIX and probably a lot of other operating systems. (Linux would be included in that list based on the comments here.)
Processor Affinity (Score:1, Informative)
Comment removed (Score:5, Informative)
Available through POSIX pthread and RT extension (Score:5, Informative)
man pset_create, pset_assign, or pset_bind.
I just checked and it seems that Solaris, HP-UX support this POSIX feature. Maybe it's only Linux that is non-standard: I found ythis quote: http://developers.sun.com/solaris/articles/solaris_linux_app.html [sun.com]
"Both Linux and the Solaris OS support the notion of binding a process or thread to a processor. Linux allows binding to a set of processors for non-exclusive use of those processors. The Solaris OS allows binding to a set of processors for exclusive use, (that is, CPU fencing), but does not allow binding to a group for non-exclusive use (except via Solaris Zones?). Linux does not have a mechanism for CPU fencing, though implementations can be found on the web (see, for example, the CPUSETS for Linux page on the bullopensource.org site). The Linux system calls that are processor affinity based are sched_setaffinity(2) and sched_getaffinity(2)."
its called "scheduler activations" (Score:5, Informative)
what is being discussed is called "scheduler activations" within the CS community (or was). its an old idea. i did some work on a real-world (hah) implemention back in the early 1990's when i worked at UWashington. google it. Solaris actually added this design at least 10 years ago (plus or minus 2 years). its a very cool OS design, but can also be hard to get the implementation right; it also requires both kernel and userspace implementations.
the basic idea is that the kernel doesn't try to decide which threads within a task/process should run. as long as the process is scheduled to have access to a CPU, whenever its about to block (e.g. on disk i/o) or to be granted a processor from another task, the kernel tells the user space scheduler what is going on. scheduling is then done in user space, where maximal knowledge about the applications internal design and thread priorities can be easily accessed.
there are several papers on this design, ranging from Tom Anderson's "original" through reports on various implementation efforts. it was certainly fun trying to write a user space context switch routine that has to be reentrant itself, not to mention trying to deal with priority inversion issues. i think sun simply worked around the latter problem with some design assumptions/limitations, but i don't know for sure.
Re:which processors (Score:1, Informative)
In other words, while a call to pthread_create() would start a new, separate thread of execution in *your* process, but may or may not start a new kernel-level thread that would run your new execution thread. And a user-level scheduler would schedule your threads on your LWPs. The kernel would schedule the LWPs.
Sun junked this because it doesn't scale well when you get into hundreds of heavyweight processes and thousands of threads and/or LWPs.
In Solaris 9 (and the "alternate thread library in Solaris 8), each separate call to pthread_create() spawns an LWP, and there's only one scheduler - the kernel's. There's no longer on in each multithreaded app. (And yes, there were ways to get this one-to-one behavior in Solaris 8 and earlier, but that's not the point...)
And this guy's saying we should go back to that?
Sorry, but Sun knows a lot more about massively parallel architecture than Intel does. Intel lives in a world where 8 CPUs is a lot.
Re:Is James clueless? (Score:2, Informative)
Comment removed (Score:4, Informative)
Re:Don't jump to conclusions (Score:3, Informative)
He's an expert at (amonst other things) providing automatically optimised applicaton parallelisation in general, which as such is not available (I can know, I used to be a reaesrch team leader in a company that was working on it as well, but from a power management angle).
What he said about OS-level schedulers is that he ideally wants them to behave in a specific way, without even saying that specifically Linux needs to be changed (again: there are hundreds of OS-es out there). He has not said that Linux is "behind" or "retarted" anything. He has not said that Intel needs volunteers to do his dirty work for free. The only person who used the word Linux during that interview was the interviewer (as usual, the /. title gets it completely wrong and the /. summary is a composition of two quotes originating from the guy's anwers to two different questions). Read and understand the bloody article before posting nonsense!
In any case, if the thing has been available for many years and people are to be ridiculed for seemingly not knowing that, they what the hell are Ingo and Con fighting about? They must be non-experts doing useless work. Both of them.
Re:Ignorant Comment (Score:2, Informative)
Re:which processors (Score:3, Informative)
By the way, NetBSD still uses an N:M model (FreeBSD tried it, but abandoned it due to code complexity issues).
Re:its called "scheduler activations" (Score:3, Informative)