Microsoft Reports OSS Unix Beats Windows XP 442
Mortimer.CA writes "In a weblog entry, Paul Murphy mentions a Microsoft report (40 page PDF) that in many instances FreeBSD 5.3 and Linux perform better than Windows XP SP2. The report is about MS' Singularity kernel (which does perform better than the OSS kernels by many of the metrics they use), and some future directions in OS design (as well as examination of the way things have been done in the past)." From the post: "What's noteworthy about it is that Microsoft compared Singularity to FreeBSD and Linux as well as Windows/XP - and almost every result shows Windows losing to the two Unix variants. For example, they show the number of CPU cycles needed to "create and start a process" as 1,032,000 for FreeBSD, 719,000 for Linux, and 5,376,000 for Windows/XP."
44 pages and the main question is still unanswered (Score:4, Insightful)
Here's an interesting snippet I found while perusing the PDF...thought I'd share. Interesting...Singularity is ostensibly supposed to be about stability, but the 44-page paper has no data on this. Kinda like saying, "Our new bulletproof vest is 40% lighter than our leading competitors, and twice as flexible. How well does it stop bullets, you ask? Sorry...we do not yet have results for that benchmark.".
Wake me when a paper comes out about Microsoft's new stability-oriented OS that actually addresses that particular aspect of the product.
Singularity is truly an intriguing system. (Score:5, Insightful)
In twenty or so years we may look back at Microsoft Research with the same admiration we have for Bell Labs.
Competition (Score:1, Insightful)
How can you hate microsoft or *nix? We need each other so that we can constantly get better and gauge our progress. I really hope that MS can learn to live with the OSS community; it will benefit everyone, including MS.
Re:44 pages and the main question is still unanswe (Score:5, Insightful)
Microsoft Research is not Microsoft. (Score:4, Insightful)
Re:What's the point of CreateProcess benchmarks? (Score:5, Insightful)
Give me a fucking break (Score:5, Insightful)
For one thing, Windows is not slower than Unix in most of the tests. It's slower than Unix in some of the tests and faster in others. For another, these benchmark results are for low-level things like spawning processes and threads. Any programmer who knows anything about Unix and Windows will tell you that threads are cheaper in Windows and processes are cheaper in Unix, because that's how they were designed. So of course Windows is going to be slower than Unix at creating processes, and of course Unix is going to be slower than Windows at creating threads.
The only thing worth reporting about this thing is the performance of Singularity, which looks like it's shaping up to be an excellent modern kernel.
Processes v. threads (Score:3, Insightful)
No I didn't RTFA.
Re:Singularity is truly an intriguing system. (Score:5, Insightful)
Typical slashdot post exaggerations (Score:5, Insightful)
Re:Too Telling (Score:2, Insightful)
I'm happy though that MS may be taking Singularity seriously. Maybe we will see their OS in 2011-2015 based on it? Unless some sort of major shift in its purpose occurs, then I would definitely jump ship from whatever I am on then, to that and I will definitely port/develop my software for the OS.
Re:Processes v. threads (Score:5, Insightful)
Exactly. NT got it's process model from VMS, and process creation was a very heavyweight operation. Unix, by contrast, had a very lightweight process creation operation. Hence NT needed threads to provide a faster alternative to processes, while Unix (whose processes were almost as cheap to create as NT threads) didn't really need threads for anything other than a marketing checklist (about the only thing Unix threads get you that processes don't is fully-shared address space, and I'd argue that's often more a problem than an advantage).
Re:Singularity is truly an intriguing system. (Score:4, Insightful)
Re:Give me a fucking break (Score:5, Insightful)
Wow is this ever misleading (Score:5, Insightful)
Here's the table from the paper, ranked best-worst, W=windows, F=freebsd, L=linux, S=singularity:
Read Cycle Counter: W: 2, F: 6, L: 6, W: 2, S: 8
ABI Call: S:87, L:437, W:627, F:878
Thread Yield: S:394, W:753, L:906, F: 911
2-Thread ping-pong: S:1207, W:1658, L: 4041, F: 4707
2-Message ping-pong: S:1452, L: 5797, W: 6244, F: 13304
Process Creation: S: 300000, L: 719000, F: 1032000, W: 5376000
The only stat in this table that Windows trails on is process creation. And anybody who has ever ported Unix code to Win32 knows exactly why: Windows is thread-oriented, and Windows systems don't tend to use helper programs or demand-forking to get work done. Which might be why Windows beats Unix in the thread benchmarks, but not in the IPC benchmarks. On the more general benchmarks, like cycles to issue a system call, Windows falls smack in the middle --- and, again, Windows has a slightly different take on what is and isn't a system call.
Drawing comparisons between Singularity and normal operating systems here is silly. Singularity doesn't have processes in the conventional sense; since there's no hardware dependencies on "process" creation in Singularity, IPC and forking are much faster.
Which is why this benchmark is reasonable inside the Singularity tech report (they're trying to demonstrate that there's a major performance benefit in rethinking boundaries between programs), but totally unreasonably outside that context: these are micro-benchmarks, like the ones CISC and RISC people throw at each other, and don't describe the amount of time it takes to complete a high-level task. Time to execute a system call is meaningful only in the context of how many system calls it takes to complete the task you're measuring.
Re:Too Telling (Score:3, Insightful)
Depends (Score:3, Insightful)
In other words, write the benchtest for the sorts of sub-category of cases that side with you, and you can make any benchmarks show what you want them to show. That's one reason they should always be viewed with a pinch of salt and a dash of vinegar, then served in newspaper.
Re:44 pages and the main question is still unanswe (Score:4, Insightful)
Re:Singularity is truly an intriguing system. (Score:1, Insightful)
that said, I agree with you that they get even more shit than they deserve. Not all of their products are terrible. MS Office is actually significantly more efficient than OOo. they've done a very nice job with MSN messenger. I can't comment on Singularity, as I know next to nothing about it. I still think they're satan incarnate, but I'm willing to give credit where credit is due.
Re:Singularity is truly an intriguing system. (Score:5, Insightful)
One other big problem from MSR - on the occasional project that's actually good, they somehow manage to kill it, or at least never tech transfer it into products. I cry when I think of some of the awesome dev technologies MSR was working on a few years ago that never made it out.
Did you actually read it? (Score:5, Insightful)
You didn't really read it, did you? From TFA(bstract).
The point of the paper is NOT to demonstrate a fully working uber-dependable system, but to validate the practicality of the architecture that is under development, and the new technologies being included. That's why they have the section on performance, with the preface (right above your quote, btw):
That's the point of the paper. I understand, however, that you might have been in too much of a rush to get first post that you didn't understand the point of the paper...
ZDNet must be very desperate... (Score:2, Insightful)
Re:That explains a lot (Score:5, Insightful)
As a result, you get tons of unstable Windows applications because to get any reasonable concurrency you have to throw out the years of hard work that OS designers put into having protected memory.
Threads vs. processes isn't "two different ways of doing the same thing". Barring a massive implementation boondoggle, you make that choice based on whether you want memory protection or not. These numbers highlight a massive boondoggle, which takes the correct choice away from the application author in many cases.
Re:Singularity is truly an intriguing system. (Score:2, Insightful)
I don't know your background, but I have never met anyone with a substantial CS background to say anything bad about MS research, including a number of OSS zealots (such as myself). I suspect that most people who make fun of them are simply not in a position to judge their work.
Re:44 pages and the main question is still unanswe (Score:3, Insightful)
My job is in QA. Your statement says that my job is impossible. Here are a few ways you can test stability:
1) See if the OS comes back online after a power cycle
2) Insert and remove device drivers
3) Send mangled data across the various data busses
4) Run programs that try to allocate all the memory
5) Run programs that try to hog all the CPU
6) Run a program that fills the hardisk/erases the hardisk/refills the hardisk
7) Do all the above all at the same time
The OS is a control point, and should be able to handle all of this and more GRACEFULLY.
(Gracefully being an undefined weasle word indicating that it should fail in a somewhat predictable manner that won't completely piss off the vast majority of administrators.)
Re:But how...? (Score:3, Insightful)
Having done a fair amount of GUI programming myself, I find a multiprocess solution is often correct (e.g. in something like Photoshop image filters, where you want shared access to one memory segment but don't need to share huge numbers of different memory segments where you can't easily compartmentalize them). For some jobs, though, multiple threads is better. Use the right tool for the job.
every Win32 process gets GUI crap at start-up (Score:3, Insightful)
UNIX creates a process with fork, which takes no arguments. UNIX runs a new executable with execve, which takes 3 arguments. So in just two system calls with 3 arguments, you launch an app.
Windows has a CreateProcess() [microsoft.com] function with 10 arguments, many of which are pointers to structs. I call your attention to the absurd "LPSTARTUPINFO lpStartupInfo " argument, which supplies info about the windows style and current desktop.
Re:Off topic: glibc updating. (Score:2, Insightful)