Slashdot stories can be listened to in audio form via an RSS feed, as read by our own robotic overlord.

 



Forgot your password?
typodupeerror
Operating Systems Software Linux Hardware

Intel Chief Evangelist Comments on Linux Scheduler 178

Posted by Zonk
from the little-penguins-everywhere dept.
ByeByeWintel writes "James Reinders is Intel's Chief Evangelist for Intel's Software Development Products. In a recent interview on Devx.com he stated: 'If I could get ONE wish fulfilled would be for OS scheduling to focus on processes, and not threads, for scheduling. And demand that processes manage their scheduling of threads ... There is a lot of opportunity for operating systems to offer these types of control in the 'running of applications' interfaces. I'd like an OS to let me specify the 'world' my application runs in (which processors, how many, etc.) These interfaces are available in Windows at run time (the task manager will let you adjust where a running task can go).'"
This discussion has been archived. No new comments can be posted.

Intel Chief Evangelist Comments on Linux Scheduler

Comments Filter:
  • by User 956 (568564) on Friday September 28, 2007 @07:58PM (#20790217) Homepage
    If I could get ONE wish fulfilled would be for OS scheduling to focus on processes, and not threads

    Yeah, a lot of us feel the same way about the fancy-dressing guys that work over in the sales office.
    • If I could get ONE wish fulfilled would be for OS scheduling to focus on processes, and not threads

      Yeah, a lot of us feel the same way about the fancy-dressing guys that work over in the sales office.

      If there ever was a case for a "+1 Brilliant" moderation option or a final moderation of (Score:6 Funny), this is it.
    • by JoelKatz (46478) on Saturday September 29, 2007 @11:42PM (#20798869)
      Brilliant.

      The idea that the scheduler should focus on processes rather than threads is one of those well-known wrong ideas that comes up all the time. The thinking is that it doesn't make sense to give twice as much processing time to a process just because it uses two threads. The thing is, does it make sense to give a job twice as much processing time just because it uses two processes rather than two threads?

      There is no reason for the scheduler to treat a thread better or worse just because of which process it belongs to. There is simply no rational defense for this position.
  • I don't understand (Score:5, Informative)

    by Junta (36770) on Friday September 28, 2007 @08:01PM (#20790235)

    I'd like an OS to let me specify the 'world' my application runs in (which processors, how many, etc.) These interfaces are available in Windows at run time (the task manager will let you adjust where a running task can go).'"
    Isn't this *exactly* what numactl gives you (hint, you don't have to run numa for numactly to allow processes to be bound to specific sets of processors).
    • Re: (Score:3, Interesting)

      by EvilGrin666 (457869)

      Isn't this *exactly* what numactl gives you
      Well yes, exactly. numactl will do this. However his bone of contention appears to be that there is no pointy clicky interface a la Windows TaskManager to do this. So I'm slightly puzzled why he's making out that there is a deficiency in the OS when the fact is that it's purely a lack of features in the window manager...
      • by Bryan Ischo (893)
        While I don't dispute in any way what you say, I'd like to point out that 'numactl' is not the most obvious terminology for a feature that applies to all platforms, NUMA or not. If it really is so general purpose as to support what the Intel guy is talking about on all platforms, then it should either be renamed, or there should at least be another interface with a more approprate name whose implementation calls into the numactl interface.

        Obviously if what you say about numactl is true, then this Intel guy
        • Re: (Score:3, Insightful)

          by xenocide2 (231786)
          Clearly, you expect too much from someone who's job title includes the word "Evangelist".
          • by Rogerborg (306625) on Saturday September 29, 2007 @04:34AM (#20792251) Homepage

            Christ, yes. Propagandist is more accurate.

            Back in the Day, Microsoft's European DirectX 'evangelist' paid a visit to our game dev studio to try to sell us on the benefits of Direct X Retarded Mode [microsoft.com]. We had our engine running with D3DRM and Glide (I said, back in the Day), and the Glide framerate was easily twice that of D3DRM. He told us that that was impossible. We said "Look at the screen.". He literally refused to look. He just insisted that we were wrong, that there was no reason that D3DRM should be slower than Glide, and that we must therefore be doing something wrong. Did I mention that he wouldn't look at the screen?

            Needless to say, we didn't even offer him lunch.

        • I'd just like to point out that numactl is merely one of several command line interfaces to that particular system call, which is actually named sched_setaffinity().
    • Re: (Score:3, Insightful)

      by timeOday (582209)
      I just hope the emphasis remains on a scheduler that "does the right thing" instead of requiring manual specification of these things to get acceptable performance for most situations. (I'm sure there are cases where manually pegging particular processes to particular CPUs is a good thing, but I can't think of any offhand).
      • Not many, but in numa, there is benefit for allocations and execution occur on the same memory 'node'. Of course, the OS scheduler should know what to deal with this...
  • So do it (Score:5, Insightful)

    by QuantumG (50515) <qg@biodome.org> on Friday September 28, 2007 @08:02PM (#20790243) Homepage Journal
    I hear Intel has some engineers who you could get to implement this. If you don't completely botch it, you might even get it into Linus' tree.

    It's Free Software, you can add whatever you want.

    • Re:So do it (Score:5, Informative)

      by Anonymous Coward on Friday September 28, 2007 @08:36PM (#20790507)
      Way ahead of you.

      $ sudo apt-get install schedutils
      ...
      Unpacking schedutils (from .../schedutils_1.5.0-1_amd64.deb) ...
      Setting up schedutils (1.5.0-1) ...

      $ man taskset | cat
      Reformatting taskset(1), please wait...
      TASKSET(1) Linux User's Manual TASKSET(1)

      NAME
      taskset - retrieve or set a processes's CPU affinity

      SYNOPSIS
      taskset [options] [mask | list ] [pid | command [arg]...]

      DESCRIPTION
      taskset is used to set or retrieve the CPU affinity of a running pro-
      cess given its PID or to launch a new COMMAND with a given CPU affin-
      ity. CPU affinity is a scheduler property that "bonds" a process to a
      given set of CPUs on the system. The Linux scheduler will honor the
      given CPU affinity and the process will not run on any other CPUs.
      Note that the Linux scheduler also supports natural CPU affinity: the
      scheduler attempts to keep processes on the same CPU as long as practi-
      cal for performance reasons. Therefore, forcing a specific CPU affin-
      ity is useful only in certain applications.

      The CPU affinity is represented as a bitmask, with the lowest order bit
      corresponding to the first logical CPU and the highest order bit corre-
      sponding to the last logical CPU. Not all CPUs may exist on a given
      system but a mask may specify more CPUs than are present. A retrieved
      mask will reflect only the bits that correspond to CPUs physically on
      the system. If an invalid mask is given (i.e., one that corresponds to
      no valid CPUs on the current system) an error is returned. The masks
      are typically given in hexadecimal. For example,

      0x00000001
      is processor #0

      0x00000003
      is processors #0 and #1

      0xFFFFFFFF
      is all processors (#0 through #31)

      When taskset returns, it is guaranteed that the given program has been
      scheduled to a legal CPU.

      OPTIONS
      -p, --pid
      operate on an existing PID and not launch a new task

      -c, --cpu-list
      specifiy a numerical list of processors instead of a bitmask.
      The list may contain multiple items, separated by comma, and
      ranges. For example, 0,5,7,9-11.

      -h, --help
      display usage information and exit

      -V, --version
      output version information and exit

      USAGE
      The default behavior is to run a new command with a given affinity
      mask:
      taskset [mask] -- [command] [arguments]

      You can also retrieve the CPU affinity of an existing task:
      taskset -p [pid]

      Or set it:
      taskset -p [mask] [pid]

      PERMISSIONS

  • BAD headline (Score:4, Insightful)

    by rmdir -r * (716956) on Friday September 28, 2007 @08:04PM (#20790261)
    Nowhere in the article does he /ever/ mention the Linux scheduler.

    Resume your regularly scheduled Con vs Ingo flamewar.

  • Oh, man (Score:5, Funny)

    by Anonymous Coward on Friday September 28, 2007 @08:04PM (#20790263)
    I scheduled an interview with linux a while back. It was all like, "LINE UP", and I was like, "WUT?", and it was like, "What's your priority?", but I was all like, "interupt, bitch", and it was like, "cool, SWAP", and I was all like, "I'm in ur processor, executin' my code".
    • Re: (Score:3, Funny)

      by $RANDOMLUSER (804576)
      "I'm in ur cache, screwin up ur prefetsh algorythm"
    • Re: (Score:3, Funny)

      by Anonymous Coward
      How long have you been waiting for a story about process scheduling so that you could post that comment?
    • Re: (Score:3, Funny)

      by ettlz (639203)
      Ahh, straight from the lolCPU pages of "I CAN HAS TIMEZLICE?".
  • I'd like an OS to let me specify the 'world' my application runs in (which processors, how many, etc.)
    Doesn't Solaris do this already?
    • Re: (Score:2, Troll)

      by mrsbrisby (60242)

      I'd like an OS to let me specify the 'world' my application runs in (which processors, how many, etc.)
      Doesn't Solaris do this already?
      So does Linux, which makes his statement moot, and indicates he is an idiot or an asshat.
    • Re: (Score:3, Informative)

      by adrianmonk (890071)

      Doesn't Solaris do this already?

      Yes, and has for at least a decade. As has SGI's IRIX and probably a lot of other operating systems. (Linux would be included in that list based on the comments here.)

    • Solaris geek time. Processor sets are a bit brute force, unless you have a very specialised workload. Placing distinct application sets into distinct projects, and then allocating those projects cpu shares for the fair shares scheduler to work with is a more general solution. Sometimes you do it to lump aspects of the same workload together, so that two workloads can co-exist reasonably. Sometimes you do it to keep aspects of the same workload together, so that one part can't swamp another. We did som
  • by MightyMartian (840721) on Friday September 28, 2007 @08:22PM (#20790397) Journal
    And I'm gonna tell ya... I'm gonna tell all the people here-a, that the Lord has come... Amen! Yes indeed, and the Glorious Pentium IV has arrive-ed! Praise be to Pentium! And lo, the sheep of the field will line down-a with the process, and the thread-a will be managed by the application!

    Oh, don't be fooled by the Devil... No-a! AMD is the sign of the Beast-a! And he shall be cast out of heaven! Raise-a your hands to the heavens and press CTRL-ALT-DELETE!
  • by headkase (533448) on Friday September 28, 2007 @08:57PM (#20790635)
    Not only is it obvious that the submitter didn't read the article but by posting it its obvious that the 'editor' didn't either! Jebus! It's about a library template that Intel provides to developers! Linux gets a passing mention on the second page but other than that coincidence the writeup has nothing to do with the real article! And then there are people tagging the article as 'clueless' - I hope they meant the submitter because if they meant the article then ironically they would have been showing their own 'cluelessness'.
    This is the first time I've felt a need to complain about /.'s integrity btw even though I've seen more articles fly-by that shouldn't have over the years.
  • by EMB Numbers (934125) on Friday September 28, 2007 @09:25PM (#20790735)
    I think that what Intel's Evangelist wants is already available through standard POSIX pthreads and the POSIX-RT extensions.

    man pset_create, pset_assign, or pset_bind.

    I just checked and it seems that Solaris, HP-UX support this POSIX feature. Maybe it's only Linux that is non-standard: I found ythis quote: http://developers.sun.com/solaris/articles/solaris_linux_app.html [sun.com]

    "Both Linux and the Solaris OS support the notion of binding a process or thread to a processor. Linux allows binding to a set of processors for non-exclusive use of those processors. The Solaris OS allows binding to a set of processors for exclusive use, (that is, CPU fencing), but does not allow binding to a group for non-exclusive use (except via Solaris Zones?). Linux does not have a mechanism for CPU fencing, though implementations can be found on the web (see, for example, the CPUSETS for Linux page on the bullopensource.org site). The Linux system calls that are processor affinity based are sched_setaffinity(2) and sched_getaffinity(2)."
    • He's not talking about binding. Or at least not only. He's talking about the OS scheduling processes and the application (i.e. user space) scheduling the threads within them. Not a new idea either, but then again he has not said the idea was new, he has only said he would want people to use it.
  • by paulbd (118132) on Friday September 28, 2007 @09:40PM (#20790805) Homepage

    what is being discussed is called "scheduler activations" within the CS community (or was). its an old idea. i did some work on a real-world (hah) implemention back in the early 1990's when i worked at UWashington. google it. Solaris actually added this design at least 10 years ago (plus or minus 2 years). its a very cool OS design, but can also be hard to get the implementation right; it also requires both kernel and userspace implementations.

    the basic idea is that the kernel doesn't try to decide which threads within a task/process should run. as long as the process is scheduled to have access to a CPU, whenever its about to block (e.g. on disk i/o) or to be granted a processor from another task, the kernel tells the user space scheduler what is going on. scheduling is then done in user space, where maximal knowledge about the applications internal design and thread priorities can be easily accessed.

    there are several papers on this design, ranging from Tom Anderson's "original" through reports on various implementation efforts. it was certainly fun trying to write a user space context switch routine that has to be reentrant itself, not to mention trying to deal with priority inversion issues. i think sun simply worked around the latter problem with some design assumptions/limitations, but i don't know for sure.

    • by anticypher (48312) <anticypher@gmai l . c om> on Friday September 28, 2007 @11:03PM (#20791175) Homepage
      it was certainly fun trying to write a user space context switch routine that has to be reentrant itself, not to mention trying to deal with priority inversion issues

      I haven't worked in that field in some number of decades, but I'm going to have nightmares because of that phrase. You heartless Bastard, at least warn us that the path to madness lies within your post.

      the AC
      sits here rocking back and forth, afraid to google what Tom Anderson's later works covered, but knowing I'm about to lose my weekend to this

  • Would a virtualization layer fulfill this? VMware's ESX is essentially an OS in itself so it has schedulers that schedule VMs, which is a lot like processes. Inside the VM, the guest OS then schedule its own processes. So maybe the answer is to write and ship application inside virtual appliances. This is just me pondering.
    • Re: (Score:2, Interesting)

      by jeremiahbell (522050)
      Putting all programs inside virtual machine like wrappers could work to keep the programmer from having to worry about scheduling in a scenario where the processes handled their threads, and it may also have good security implications. A major problem is malicious programs gaining access to the RAM, and trying to ferret our your hard-drive encryption key for example. Putting all programs in a locked Box might be a great way to fix the problem. Despite the performance penalty I believe this may be the fut
  • by snowwrestler (896305) on Saturday September 29, 2007 @10:00AM (#20793415)
    If I had one wish that I could wish this holiday season, it would be for all the children of the world to join hands and sing together in the spirit of harmony and peace.

    If I had two wishes that I could wish for this holiday season, the first would be for all the children of the world to join hands and sing in the spirit of harmony and peace.. and the second would be for $30 million a month to be given to me, tax-free in a Swiss bank account.

    You know, if I had three wishes that I could make this holiday season, first, of course, would be for all the children to get together and sing . . . the second would be for the $30 million every month to me . . . and the third would be for all encompassing power over every living being thing in the entire universe.

    And if I had four wishes that I could make this holiday season, first would be the crap about the kids . . . second would be for the $30 million . . . the third would be for all the power.. and the fourth would be to set aside one month each year for an extended 31-day orgasm, to be brought about slowly by Rosanna Arquette and that model Paulina somebody, I can't think of her name, of course my lovely wife could come, too. She's behind me 100% on this, I guarantee you.

    Wait a minute, maybe that sex thing should be the first wish! So, if I made that the first wish, because, you know, it could all go boom tomorrow, and then what have you got? No, no . . . the kids singing would be great, that would be nice. No, no, who am I kidding! I mean, they're not gonna be able to get all those kids together! I mean, the logistics of the thing is impossible! It's more trouble than it's worth!

    So, we reorganize: here we go. First, the sex - we go with that; second, the money. No! We go with the power second, then the money, and then the kids. Oh, wait, oh geez! I forgot about revenge against my enemies! Okay . . . revenge against all my enemies, they should die like pigs in Hell! That would be the fourth wish! And of course, my fifth wish would be for all the children of the world to join hands and sing in the spirit of peace and harmony.

    Thank you, everybody.

Old programmers never die, they just branch to a new address.

Working...