Intel Chief Evangelist Comments on Linux Scheduler 178
ByeByeWintel writes "James Reinders is Intel's Chief Evangelist for Intel's Software Development Products. In a
recent interview on Devx.com he stated: 'If I could get ONE wish fulfilled would be for OS scheduling to focus on processes, and not threads, for scheduling. And demand that processes manage their scheduling of threads ... There is a lot of opportunity for operating systems to offer these types of control in the 'running of applications' interfaces. I'd like an OS to let me specify the 'world' my application runs in (which processors, how many, etc.) These interfaces are available in Windows at run time (the task manager will let you adjust where a running task can go).'"
the real world, too. (Score:5, Funny)
Yeah, a lot of us feel the same way about the fancy-dressing guys that work over in the sales office.
Re: (Score:2, Funny)
Yeah, a lot of us feel the same way about the fancy-dressing guys that work over in the sales office.
If there ever was a case for a "+1 Brilliant" moderation option or a final moderation of (Score:6 Funny), this is it.
Re: (Score:2)
Re:the real world, too. (Score:4, Insightful)
The idea that the scheduler should focus on processes rather than threads is one of those well-known wrong ideas that comes up all the time. The thinking is that it doesn't make sense to give twice as much processing time to a process just because it uses two threads. The thing is, does it make sense to give a job twice as much processing time just because it uses two processes rather than two threads?
There is no reason for the scheduler to treat a thread better or worse just because of which process it belongs to. There is simply no rational defense for this position.
I don't understand (Score:5, Informative)
Re: (Score:3, Interesting)
Re: (Score:2)
Obviously if what you say about numactl is true, then this Intel guy
Re: (Score:3, Insightful)
Re:I don't understand (Score:5, Interesting)
Christ, yes. Propagandist is more accurate.
Back in the Day, Microsoft's European DirectX 'evangelist' paid a visit to our game dev studio to try to sell us on the benefits of Direct X Retarded Mode [microsoft.com]. We had our engine running with D3DRM and Glide (I said, back in the Day), and the Glide framerate was easily twice that of D3DRM. He told us that that was impossible. We said "Look at the screen.". He literally refused to look. He just insisted that we were wrong, that there was no reason that D3DRM should be slower than Glide, and that we must therefore be doing something wrong. Did I mention that he wouldn't look at the screen?
Needless to say, we didn't even offer him lunch.
That was intended (Score:2)
Exactly the point and apparently, according to the poster, the Microsoft drone kept pretending that :
Re: (Score:2)
Re:The name doesn't need to be obvious (Score:5, Funny)
Well, sure. 'ls' gives you a catalog of files on the disk, while 'cat' lists one or more of the files' contents. But Windows' 'dir' is much harder to remember - it just means "Drrr, I wonder what's in this folder?"
Re: (Score:2)
anyway, it is not windows 'dir' but dos 'dir' and there were no folders in dos, only directories.
cmd.exe is a Win32 application (Score:2)
anyway, it is not windows 'dir' but dos 'dir'
As of Windows 2000 and Windows XP, it is Windows dir. The program cmd.exe that interprets commands such as dir is a Win32 application, not a 16-bit MS-DOS application or a 32-bit DPMI application. Open %systemroot%\system32\cmd.exe in a hex editor to see the ASCII string "This program cannot be run in DOS mode."
there were no folders in dos, only directories
Are there "files" and "folders" in the Spanish version of Windows, as opposed to "archivos" and "carpetas"? As I understand it, "folder" is the English-locale counterpart of what is called a "direc
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Insightful)
In non-numa cases.. (Score:2)
So do it (Score:5, Insightful)
It's Free Software, you can add whatever you want.
Re:So do it (Score:5, Informative)
Re: (Score:2)
Re:So do it (Score:5, Insightful)
Re:You think maybe they have other things to do? (Score:5, Insightful)
Re:You think maybe they have other things to do? (Score:5, Insightful)
Re: (Score:2)
An interesting point. I would say he's definitely trying to get someone to step up, and Intel does have the people who could do just that... (be good for the multi-core market)
Re: (Score:2)
Re: (Score:3, Interesting)
Yes, but if you have to pay someone to make the modifications you want done, and then pay them to maintain those modifications and to port them whenever something they interface with randomly changes, then you've got cost, probably quite substantial cost
Yes. People, on a large scale, don't work for free. Especially talented ones capable of correctly implementing something like per-cpu scheduling hints. What you can do to minimize the cost is pay people to get it into upstream. Moreover, the source code availability in the first place vastly increases competition. It's much easier to patch linux than to design an operating system people want to run software on that also happens to provide scheduler tweaks you like. In the closed source system you're negot
The problem is (Score:2)
For people it is twofold. The first is not knowing even how to go about it. You really think a non-technical user knows how to go about having software or features written? Hardly. They wouldn't know who to hire. The second problem is one of money. I'd love to have a nice OSS audio/video editor written. Basically I'd like to have most of Sony's Vegas and some of Cakewalk's Sonar. That's be just great, I'd even be willing to pay. The problem is I wouldn't be
Re: (Score:2)
For people it is twofold. The first is not knowing even how to go about it. You really think a non-technical user knows how to go about having software or features written? Hardly.
This hardly only applies to open source software. How do you go about getting Microsoft or Apple to include your needs or fixes? You don't. You pick among the options they offer, and maybe offer feedback in beta testing (which few nontechnical types do). Closed source software is at the mercy of a single product market -- if you're lucky, the developer recognizes that a particular feature you want is among the highest demand features and works on it. They have limited ability to capture your additional ben
Re: (Score:2)
Re: (Score:2)
It's like buying furniture, if you want one that exactly fits your needs you need to build it yourself or find someone to do it for you.
Now as I said you can't respond to each individual request. However you aggregate it to the whole community of users, and you can start to get an idea of what you need to focus on, and the rules still hold true: Implementing things people want keeps users and gets new one.
What the fuck do you THINK OSS developers do? Hold crack parties throughout the night and use a dart board to think of new features in their
Re: (Score:2)
Re: (Score:2)
While that's true, in the case of chipping in on a bounty to get something written you *might* get suitable software *eventually*. If you spend the money on a proprietary solution instead, then (as long as a suitable product exists
Re: (Score:2)
Re: (Score:3, Insightful)
Re:Ok, that's completely fine (Score:4, Insightful)
As for considering people's ideas, yeah, great, developers do that, all the time. The problem is when the developer says "look, it won't work" and the people harp on about how right they are. That's often when the DIY attitude comes out, because nothing says "you were wrong" like running code.
Re: (Score:2)
On the other hand, there is no shortage of people complaining that a feature that they want is not present in a piece of software, yet expect someone else to write it for them. Or demanding "Linux isn't ready for
Re: (Score:2)
Indeed, and surely a comment that encourages the poster to get involved in the project (i.e. the Linux kernel) and explore their ideas of where it should change is the opposite of elitist. Elitist would be "it's good enough already, and it's not your project so push off or else".
Re:You think maybe they have other things to do? (Score:4, Insightful)
You expect every fucking suggestion to be implemented? If not then what DO you suggest? The developers simply say that they are going to do X, if you want to do Y then you need to do it yourself. You can't implement every bloody possible feature in the damn universe, which is apparently what you want. Instead they implement what they think meets their own goals. They claim that this is enough to work for everyone. It isn't perfect for everyone but NOTHING ever is, if someone wants a feature desperately they either need to implement it themselves or find someone else to do it.
Re: (Score:3, Insightful)
It's great. That's what open source is all about.
>Part of having a successful product is listening to what people want and working to implement that.
Ah I see where you got confused.
Linux is not a product. It's already successful.
Re: (Score:2)
You know what's worse?
Ungrateful fucks who cry because other people don't do things for them.
To me those are the most despicable people on the planet.
Re: (Score:2)
Sorry, but the "Do it yourself," attitude is just bad. Even assuming it is directed at the extremely small segment of the population that has the level of programming knowledge required (usually it isn't) that then assumes that they have nothing better to do with their time. Sorry, not how it works.
Writing from the point of view of someone who frequently receives feature requests for free software:
When someone asks the kind of question that gets a response "do it yourself" they're probably making the following implications: The developers have nothing better to do, and are just code monkeys waiting for ideas; you know what the developers should be doing better than the developers; the developers owe you something because you're using their software; the developers care whether or not you use their s
Re: (Score:2)
Come on... It's not that Linux users are elitist, arrogant bastards. It's that a lot of computer users never quite get that you cannot expect some software to form out of thin air. If you want something you can't do yourself, you may could and wait until someone who can do it also wants to do it. If you really need it, it would be best to pay someone to do it right away.
Really a sad comment on society (Score:4, Insightful)
You are right. The way free software works is that if you want something done, you either do it yourself, or you pay somebody to do it for you.
Somehow, this has become at odds with mainstream society. People have come to expect that if you want something done, you whine about it for long enough, and someone else will do it for you.
It is a society of builders and whiners.
I know which side I'm on.
Re: (Score:2)
More appropriately these days, I think it's 'if you want something done, odds are it's already been done, someone is already doing it, or there is some very good reasons your desire is either unfeasible or too unspecific'.
Arrogant is to think you're the first to want something done. Usually that just means you havent done your homework.
"I know which side I'm on."
Yep, same here.
Don't jump to conclusions (Score:2)
Nope, you complainers got it backwards. This is a tech expert in his area, saying that he would want OS developrs - which he is not - do change their ways. Being the expert that he is, he is very much entitled to say that. What's more, he would still be entitled to say that even if he were in charge of the OS. Note that he doesn't say: "go out and do it for me because I'm lazy and don't want to pay". For all we know, he might be busy pulling "evangelical" levers to get it done at Intel and we are just obse
Re: (Score:3, Informative)
He's an expert at (amonst other things) providing automatically optimised applicaton parallelisation in general, which as such is not available (I can know, I used to be a reaesrch team leader in a company that was working on it as well, but from a power management angle).
What he said about OS-level schedulers is that he ideally wants them to behave in a specific way, without even saying that specifically Linux needs to be changed (again: there are hundreds of OS-es out there). He has not said that Linux
Re: (Score:2)
Come on, he's not saying that you can use C++ templates in C. They have implemented a user space library to support user space steared inter-process parallelization and scheduling. If you want to use it, you have to use C++ for writing your app, but it does not have to be inside the OS. What he wants the OS to do, is to keep out of the way of what his liubrary does by scheduling processes as a whole instead of single threads.
(correction to critical typo) (Score:2)
Re: (Score:2)
1: Where does iut say that Intel is demanding a bunch of volunteers to do their dirty work?
2: How do you know what Intel is doing? Do you work for them? Or do you at least work with them on such topics? Or do you at the very least work in the same area (I personally did so until a little over a year ago)?
3: Why should they at all cost try to implement a new scheduler when the Linux community is already fighting over two of them, written by people who are being paid for it? Better to explain what they thi
Re: (Score:2)
Re: (Score:2)
kthxbye.
BAD headline (Score:4, Insightful)
Resume your regularly scheduled Con vs Ingo flamewar.
Re: (Score:2)
Sorry, I couldn't resist...
Re: (Score:2)
Re: (Score:2)
Chris Mattern
It's in the footnotes (Score:2)
Oh, man (Score:5, Funny)
Re: (Score:3, Funny)
Re: (Score:3, Funny)
Re: (Score:3, Funny)
which processors (Score:2)
Re: (Score:2, Troll)
Re: (Score:2)
You think it's possible that he's not asking for CPU (and possibly IRQ) affinity settings?
I'd assume that's what he's asking for because that is what's in the task manager, but you're right, I might not understand what he's asking for.
However, given that he says (8-core machine--run "only Outlook" here, run applications on these 4 cores, OS only here, explorer here, etc.) I think that infact I do know what he's talking about. It's a bit muffled from talking
Re: (Score:2)
What gave you that idea? He said 8-core machine--run "only Outlook" here, run applications on these 4 cores, OS only here, explorer here, etc - that sounds exactly like a naive explanation of CPU affinity.
His job? He's the "Chief Evangelist Officer" - he's a paid fanboy. H
Re: (Score:3, Informative)
Yes, and has for at least a decade. As has SGI's IRIX and probably a lot of other operating systems. (Linux would be included in that list based on the comments here.)
Re: (Score:3, Informative)
Sun junked this because it doesn't scale well when you get into hundreds of heavyweight processes and thousands of threads and/or LWPs.
I think you are mistaken as to the reasoning. This model does scale well to large numbers of threads; much better than the 1:1 model. The problem is, C programmers don't write code that spawns thousands of threads, because C is not suited to concurrency. C programmers tend not to create more than a dozen or so threads, and so the added complexity of N:M is not needed because 1:1 can handle it well. Languages like Erlang (and, to a much lesser degree, Java), which do encourage high orders of parallelism
Re: (Score:2)
that the new threading model just plain performed better (;-))
--dave (who was a performance engineer at Sun at the time) c-b
Processor Sets are often the Wrong Answer (Score:2)
Intel's Chief Evangelist (Score:5, Funny)
Oh, don't be fooled by the Devil... No-a! AMD is the sign of the Beast-a! And he shall be cast out of heaven! Raise-a your hands to the heavens and press CTRL-ALT-DELETE!
Re:Intel's Chief Evangelist (Score:4, Funny)
(Reboot!)
Slashdot would be kicked out of Journalism School (Score:5, Insightful)
This is the first time I've felt a need to complain about
Re:Slashdot would be kicked out of Journalism Scho (Score:2)
Re:Slashdot would be kicked out of Journalism Scho (Score:2)
People read slashdot for comments.
People read digg for amusing pictures and pop culture things.
Editors are people too.Unlike userdriven selection of stories,it depends on one man choosing whatever on his mind at the time.
Re:Slashdot would be kicked out of Journalism Scho (Score:2)
New here?
Available through POSIX pthread and RT extension (Score:5, Informative)
man pset_create, pset_assign, or pset_bind.
I just checked and it seems that Solaris, HP-UX support this POSIX feature. Maybe it's only Linux that is non-standard: I found ythis quote: http://developers.sun.com/solaris/articles/solaris_linux_app.html [sun.com]
"Both Linux and the Solaris OS support the notion of binding a process or thread to a processor. Linux allows binding to a set of processors for non-exclusive use of those processors. The Solaris OS allows binding to a set of processors for exclusive use, (that is, CPU fencing), but does not allow binding to a group for non-exclusive use (except via Solaris Zones?). Linux does not have a mechanism for CPU fencing, though implementations can be found on the web (see, for example, the CPUSETS for Linux page on the bullopensource.org site). The Linux system calls that are processor affinity based are sched_setaffinity(2) and sched_getaffinity(2)."
Re:Available through POSIX pthread and RT extensio (Score:2)
its called "scheduler activations" (Score:5, Informative)
what is being discussed is called "scheduler activations" within the CS community (or was). its an old idea. i did some work on a real-world (hah) implemention back in the early 1990's when i worked at UWashington. google it. Solaris actually added this design at least 10 years ago (plus or minus 2 years). its a very cool OS design, but can also be hard to get the implementation right; it also requires both kernel and userspace implementations.
the basic idea is that the kernel doesn't try to decide which threads within a task/process should run. as long as the process is scheduled to have access to a CPU, whenever its about to block (e.g. on disk i/o) or to be granted a processor from another task, the kernel tells the user space scheduler what is going on. scheduling is then done in user space, where maximal knowledge about the applications internal design and thread priorities can be easily accessed.
there are several papers on this design, ranging from Tom Anderson's "original" through reports on various implementation efforts. it was certainly fun trying to write a user space context switch routine that has to be reentrant itself, not to mention trying to deal with priority inversion issues. i think sun simply worked around the latter problem with some design assumptions/limitations, but i don't know for sure.
Re:its called "scheduler activations" (Score:5, Funny)
I haven't worked in that field in some number of decades, but I'm going to have nightmares because of that phrase. You heartless Bastard, at least warn us that the path to madness lies within your post.
the AC
sits here rocking back and forth, afraid to google what Tom Anderson's later works covered, but knowing I'm about to lose my weekend to this
Re: (Score:3, Informative)
Virtual Machines? (Score:2)
Re: (Score:2, Interesting)
Obligatory Steve Martin (Score:3, Funny)
If I had two wishes that I could wish for this holiday season, the first would be for all the children of the world to join hands and sing in the spirit of harmony and peace.. and the second would be for $30 million a month to be given to me, tax-free in a Swiss bank account.
You know, if I had three wishes that I could make this holiday season, first, of course, would be for all the children to get together and sing . . . the second would be for the $30 million every month to me . . . and the third would be for all encompassing power over every living being thing in the entire universe.
And if I had four wishes that I could make this holiday season, first would be the crap about the kids . . . second would be for the $30 million . . . the third would be for all the power.. and the fourth would be to set aside one month each year for an extended 31-day orgasm, to be brought about slowly by Rosanna Arquette and that model Paulina somebody, I can't think of her name, of course my lovely wife could come, too. She's behind me 100% on this, I guarantee you.
Wait a minute, maybe that sex thing should be the first wish! So, if I made that the first wish, because, you know, it could all go boom tomorrow, and then what have you got? No, no . . . the kids singing would be great, that would be nice. No, no, who am I kidding! I mean, they're not gonna be able to get all those kids together! I mean, the logistics of the thing is impossible! It's more trouble than it's worth!
So, we reorganize: here we go. First, the sex - we go with that; second, the money. No! We go with the power second, then the money, and then the kids. Oh, wait, oh geez! I forgot about revenge against my enemies! Okay . . . revenge against all my enemies, they should die like pigs in Hell! That would be the fourth wish! And of course, my fifth wish would be for all the children of the world to join hands and sing in the spirit of peace and harmony.
Thank you, everybody.
Re:Puh-leeeeze! (Score:4, Insightful)
Comment removed (Score:5, Informative)
Re:Puh-leeeeze! (Score:5, Funny)
Wait a minute, is that a compliment or an insult?!?
Comment removed (Score:4, Informative)
Re: (Score:2)
Re: (Score:2)
[...] and even the X-serve trumps similar servers using the same chips from Intel.
How's that, exactly ? They're more expensive, have shorter warranties, fewer support options, fewer configuration options and a lower-specced top-end configuration. This is before getting into the bits of the commodity server market that Apple simply has no product to cater for (low-end 1U machines, 2U machines - I ignore 4u+ machines and Blades because they're relatively uncommon).
The Xserve is OK if you want a rackmount
heh? (Score:2)
How would you be marketing a stuff you are not a fan of?
And don't tell me Apple doesn't pay well its most talented technical professionals!
Re: (Score:2)
Re: (Score:3, Interesting)
I worked with the Evangelists in Apple Developer Relations, and my direct personal experience tells me that you're full of shit.
No, he's not. The fact that some of those you've met were good does not mean it does not imply 'highly paid fanboy'. Or maybe you were questioning the 'highly paid' part :).
For balance, the only Apple Evangelist I ever had experience with was a Quickdraw 3D Evangelist. He was a clueless idiot. I asked him how they addressed the performance issue caused by QD3D lighting only working with full RGB, whereas DirectX supported a ramp mode for significant performance increase. He didn't have a fecking cl
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Informative)
Re:Hello?? (Score:4, Insightful)
Re: (Score:2)
Re: (Score:2, Informative)
Re: (Score:2, Informative)