Linux Kernel Benchmarking: 2.4 vs. 2.6-test 293
frooyo pastes from kerneltrap: "Cliff White recently posted some re-AIM multiuser benchmark results comparing the stable 2.4.23-pre5 kernel against the 2.6.0-test5 and 2.6.0-test5-mm4 development kernels. In his conclusion he makes reference to earlier scheduler tests posted by Mark Wong saying, "Short summary: we mostly rock.""
Comment removed (Score:5, Funny)
Re:Real world please. (Score:2, Funny)
Re:Real world please. (Score:4, Insightful)
Just becaues you can't see its use outside of a toy, doesn't mean everybody can't.
Re:Real world please. (Score:2, Informative)
Re:Real world please. (Score:2)
SMP (Score:5, Interesting)
That will help everyone from the server market, to me when I save up enough for a two processor motherboard.
Linux sorta Scales, but the hardware doesn't... (Score:5, Insightful)
You do need a scalable OS to suport lots of processors, of course, but you also need hardware that scales too (clustering doesn't count). Example - SGI is using Linux with NUMAflex on the Altixes [sgi.com] to cluster 64-processor system images, but that kind of hardware isn't commodity in any way, and isn't going to be anytime soon.
Anyway, Linux doesn't scale THAT well...as of 9/2000, SGI was using IRIX for a 1024-processor single-system-image supercomputer [sgi.com]; I've heard they can go to 2048 now, but I don't have anything to back that up. Dunno about Solaris, but I imagine it's pretty scalable as well.
Re:SMP (Score:4, Insightful)
I've got nothing against Linux improving at SMP in essence, but there is something very bad going on here it seems to me. Notice that while the new kernel 'kicks ass' on SMP systems, on uniprocessor systems the 2.4 kernel is the one kicking ass. Anyone benchmarked 2.4 against some of the pre-SMP kernels on a uniprocessor machine?
Face it, the vast majority of users are uniprocessor, and kernel performance is more of an issue on lower-end machines. Improving performance on big multiprocessor boxes is fine by itself, but not when it harms uniprocessor performance. I'm not a kernel hacker, but I've read many people that this would not happen, that the SMP code would not hurt performance on a uniprocessor machine when the kernel is compiled without it, but that's obviously not turning out to be the case. Anecdotal evidence, at least, suggests that this performance degradation has actually been going on for quite some time, at least back to when SMP code first started being added.
I'm not sure what all the factors here are, so naturally I'm not going to tell you the solution, but it certainly looks like a potential problem that should be discussed. Hopefully someone with more specifics than I have can chime in...
Re:SMP (Score:4, Informative)
Yeah, they missed an important test - latency for interactive processes. A lot of scheduler work went into improving this, and it makes a huge difference when you have large memory processes working hard.
This aspect is improved across the board in 2.6, as well as the SMP issues. Sure, the uniprocessor machine may be a little slower, but response latencies in X are a lot better, and this makes more of a difference to users.
Re:SMP (Score:3, Interesting)
I beg to differ. I couldn't care less about X, I haven't even used it in over a year. Like the vast majority of Linux boxes, mine runs in console mode only, and on a single processor. I, and the rest of the world that uses it like this, find it hard to see anything to get excited about i
Re:SMP (Score:2)
Re:SMP (Score:2, Funny)
I don't understand...scalability to hundreds of CPUs will provide much penis enhancement for geeks everywhere (even the ladies).
Re:SMP (Score:4, Insightful)
Obviously the SMP performance has been improved, and there was a lot of potential for improvements looking at the 8x test. Another way to interpret the results would be to say that the other changes decreased performance across the board on SMP and Uniprocessor systems. The SMP improvements in SMP machines more than made up for this added cost and improved raw performance on SMP machines.
Hopefully the performance loss on Uniprocessor machines can be decreased or eliminated. Even if it's not, I think you need to remember that raw performance isn't the be-all-end-all thing that's important. 7% is pretty small in the grand scheme of things where processing speed is doubling every 18 months. Responsiveness and better scheduling that doesn't starve processes is more important than a 7% performance decrease IMO, and you don't get that from faster processors.
Re:SMP (Score:2)
Well, why not support SMP by default? After all, Intel is going that direction with their uni-CPUs -- making them appear as two. Perhaps that is their permanent direction.
If so, then shouldn't everything be written assuming it will run on two processors, since that's all we'll have in a few years?Re:SMP (Score:2)
Oh wait...
Re:SMP (Score:4, Interesting)
Worth noting, however, it's not uncommon, even for a system that fully supports HT, to see a noteworthy performance drop when HT is enabled. Seems many new systems come with HT disabled in the BIOS for this very reason. Granted, I'm not 100% that's not a Window's specific issue rather than a broad-board HT issue, but something to keep in mind, nonetheless.
Re:SMP (Score:2, Informative)
GREAT (Score:5, Funny)
Re:GREAT (Score:2, Funny)
Re:GREAT (Score:2, Interesting)
OT: Re:GREAT (Score:2)
novel idea. (Score:4, Funny)
Re:novel idea. (Score:3, Interesting)
Which just reaffirms my belief that linux is becoming ever more firmly planted in the server world, and desktop linux is still just a hobby for the most part.
Re:novel idea. (Score:2, Insightful)
Re:novel idea. (Score:2)
Re:novel idea. (Score:5, Insightful)
While it's true that they are working hard to significantly improve Linux for the server room, by far, they have never lost site of the uniprocessor user. Remember, there is nothing wrong with tuning the kernel for your uniprocessor needs, and specific workloads. They just can't do that when they are benchmarking because it would skew the results, invalidating them. They are not only trying to measure how their improvements effect the overall system, but, what makes for sane initial defaults, which are reflective of a general purpose and broad workload. If you understand what you are doing, there is not a reason to believe that you can't greatly improve things for your specific uses and workloads. It's important to keep all of these in mind when talking about these benchmarks. Furthermore, you should fully expect your favorite distro to come with tuning presents which reflect a targeted workload (file/print server, workstation, database, web server, etc.).
Keep in mind that the benchmark you looked at represents one category of many different types of workloads. So, for that specific workload, it may of been slower, however, that workload my not represent anything you do with your computer. Remember, other types of workloads are significantly faster. One last note, remember, performance is the classic trade off with lower latencies. It trades responsiveness for raw throughput. If, on a uniprocessor workstation, you only see a -7% drop in performance and latency is greatly reduced, chances are, not only will you never notice the loss in performance, but you'll be praising it for how well it works with your mouse, monitor and keyboard (if feels better and makes you a happier user).
Just some food for thought.
Re:novel idea. (Score:4, Insightful)
One benchmark used for Linux kernels is hammering a system while playing an mp3 to see if they can get it to skip. Low latency is mostly a desktop feature, and the 2.6 kernel is going to have much improved latency.
Other portions of Linux have changed, and may not initially outperform 2.4, but if you think this kernel isn't going to be a dramatic improvement over 2.4 for desktop users and servers, and if you think the kernel developers aren't taking the desktop into consideration, you are mistaken.
Re:novel idea. (Score:3, Informative)
Re:novel idea. (Score:2)
woo (Score:5, Funny)
If you thought SCO was mad over 2.4, just wait until they make up evidence for the 2.6 kernel!
SCO Kernels (Score:5, Funny)
DARL: So, um, hey. It looks like there's this new "too-pointe-six colonel" out on the market from those Lenn-ucks people. We own all that too, right?
SUIT: Well, sir, it's like this. Do you remember how the 2.4 kernel had all of those lines of code in them that are ours, even though they showed up in textbooks before most of our stuff existed?
DARL: Sure, but how does that help us with this new thing?
SUIT: Think about it. Most operating systems, according to my extensive research during years of never having looked at a computer before, contain the same code that they always did, plus a couple of lines of new comments and an extra variable or two that shows how much you're able to charge users for the new features. Just think about the Windows 95 and 98 thing. Perfect example there.
DARL: But...my mansion only has 93 windows. Where is this heading?
SUIT: *blinks* Errr...yeah. Well, it's all the same code, and even those sneaky Linux commies try to pull a fast one on us and put one of those different codes in there, we can always assert our ownership of these "opened sources" files that I just printed out. I asked this guy, you know, and he said that all of these sources are what's in Linux, and since I printed it on paper and stuff, I figure it must be a textbook. Since we own all the words that show up in textbooks, and this has a lot of words, I think we've found ourselves a new angle here.
DARL: Smithers, cry havoc and let slip the Lenn-ucks colonel lawsuit monkeys once more!
I do so hate having to correct you people. *sigh*
Not to be a n00b... (Score:2, Insightful)
Re:Not to be a n00b... (Score:2)
Re:Not to be a n00b... (Score:5, Funny)
You actually READ the article?!? Man! You ARE a N00b!
Re:Not to be a n00b... (Score:2)
heh. A response to a "n00b" by someone who's even "newer" to /.
Re:Not to be a n00b... (Score:2)
Re:Not to be a n00b... (Score:2, Informative)
timeline? (Score:2, Insightful)
are really developers waiting in line? (Score:2)
I can only think of 2 justifiable reasons for this:
1) Developers can't figure out how to install a beta or alpha kernel..
2) Developers dont trust it enough to belive that code written for a beta will work on an official release.
Rock? (Score:5, Insightful)
Whereas it is 7% slower if you have one processor.
I suppose they'll have uniprocessor version which runs faster? Lots of people have uniprocessor pcs.
Hyperthreading doesn't really count.
Re:Rock? (Score:2, Interesting)
Re:Rock? (Score:4, Interesting)
Re:Rock? (Score:2)
Re:Rock? (Score:2)
I'm trying not to read too much into this benchmark. The new kernel preemption in 2.6 will make Linux "feel" faster even though it may be slower given a long running continuous task to chew on.
To counter balance that, I'm assuming that the focus right now is on stability rather than optimization. I'd hope that any performance gap with the 2.4 series would be closed shortly after the 2.6.0 release. What was the situtation like between 2.2
Half rock (Score:3, Insightful)
Re:Half rock (Score:5, Insightful)
The 2.6x series kernels will be a big step up for just about everyone that seriously uses their computer. Significant realiability improvements as well as faster thoughput on disks, much, much higher scalability for SMP (hyperthreading and numa and even highly loaded uni-systems) systems, and much lower latencies, all at the same time. Granted, there are still some tests which may not be a win-win all the way around, however, almost everything in general is an improvement with hardly any detracters.
So, saying, "we mostly rock", really is a true statement!
Re:Rock? (Score:2)
I've thought for a long time that you want different schedulers available for different scales of multiprocessing. Heck, even Windows has different "drivers" for uniprocessor and dual processor machines for W2K Pro.
Re:Rock? (Score:2)
Anyone notice?
Re:Rock? (Score:2)
I'm not exactly sure what you're asking? You asking if you get a different animal if you compile out the preempt code too? Meaning, higher performance at the cost of higher latency? AFAIK, the answer is yes.
> Also with hyperthreading, psuedo-SMP is really a lowend feature now.
The problem with that is, enabling SMP support, of which HT is part of, requires additional locking and more overhead to keep things straight in the kernel. To makes matters wo
Re:Rock? (Score:2)
User Experience (Score:5, Informative)
Just my observation
-the_crowbar
Re:User Experience (Score:2)
There's very little window shaking and mouse slowdowns. Still, its not resposive as (*gasp*) Windows, but its getting there.
Hopefully, 2.6.0-testx will get better.
Re:User Experience (Score:2)
Re:User Experience (Score:2)
Re:User Experience (Score:2, Informative)
I'm running in IDE mode only with no issues.
Better comparision (Score:3, Informative)
Re:Better comparision (Score:2)
Re:Better comparision (Score:2)
When we did some speed tests on Solaris x86 vs the early 2.4.x kernels about 2 uears ago (a while granted), Solaris x86 was a DOG. They may be trying to clean it up now, but even then they admitted it was basically an afterthought.
I've seen a TON of different hardware / OS configs, but I know of only one shop who used Solaris x86. They used in dual CPU machines only.
Maybe I'm wrong, but my impression
2.4 vs 2.6 (Score:3, Interesting)
This struck me as strange, because when the kernel is compiled without SMP support, all that code is left out. So it doesn't seem like the 2.4 should outperform the 2.6 on one cpu.
Does anyone know why this might be?
Re:2.4 vs 2.6 (Score:2)
Re:2.4 vs 2.6 (Score:2)
because of this but might actually run slower.
Re:2.4 vs 2.6 (Score:3, Informative)
Of course, there is the possibility of trimming cycles from the process of switching contexts. Linux, though, already had that pretty low. That's why Linus is so resistant to shared-memory, shared-context threads: the cost of processes is so low that
Re:2.4 vs 2.6 (Score:2)
my own experience with test5 shows me that it has issues with memory management/swap. the machine was completly unuseable when all the memory was used up
Re:2.4 vs 2.6 (Score:2)
This is largely due to people finding way to measure the sorts of performance you actually care about for desktops. This is at the expense of ot
Thanks SCO. (Score:5, Funny)
Now we can appriciate the forsite that our Unix fathers had when developing Xeon SMP code in the late 1970's.
I'm a bit leery. (Score:5, Interesting)
For some reason, the scheduling seems to get more and more choppy (in that i've noticed) with every iteration of 2.4.x kernel. Currently i'm on 2.4.22, and while i don't have any specific tests, numbers or statistics i'm noticing some issues.
Easiest way to reproduce it is to have the machine do something cpu intensive, such as mkisofs, cdrecord, bzip2 some huge file, cp anything large, installing (via aptitude) or even the "Reading Package Lists...." stage of apt-get update.
Oftentimes, the machine will become unresponsive for about 3 seconds at a time, then jolt back up to speed, then pause for 3, on and on. Even after the command line returns the prompt, or gkrellm's cpu and proc krells show that everything is all done, i will still see lag in responses from the kb, mouse, or whatnot off and on for about 10-15 seconds.
I've gone over my kernel config and tweaked a few things here and there but with no change. I can back down to a 2.4.18 kernel and it's not as bad. Going down to a 2.2.x kernel completely solves the problem, but of course will bring its own issues with some of my newer packages (such as gcc) and a few pieces of newer hardware.
A friend of mine and I have gone over this (on my machine and his) and he experiences a lot of the same issues i do.
Mind you, i'm not complaining. I'm very grateful to all the developers of the world that i even *have* a linux system to run. But this is something that makes me more excited about the kernel 2.6.x series. I haven't tried one out yet, but from what i've heard and read, it should be awesoe.
Re:I'm a bit leery. (Score:5, Informative)
Re:I'm a bit leery. (Score:2)
Yep... the base install is Debian Testing, and everything else was pulled out of "Unstable". I have enabled DMA, and it actually did make what appeared to be a slight difference.
However, other things that don't really use the hdd much (i.
Smoother scheduling in 2.6.0 (Score:5, Informative)
Of course, the ACPI support and swsusp doesn't hurt either
Re:Smoother scheduling in 2.6.0 (Score:2)
Re:I'm a bit leery. (Score:2)
What I wouls really like is for Andrew Morton's MM patches to be adopted. They bring in proper asynchronous I/O support and that will eventually dramatically help response times (however only as utilities get rewritten to adopt this). However. the improvements to X responsiveness seem to be here now.
Architeture (Score:3, Interesting)
MPPE build in 2.6?? (Score:2, Troll)
Re:MPPE build in 2.6?? (Score:2)
how many users would benefit from having that as
Re:MPPE build in 2.6?? (Score:2)
Strewth, I am getting on with life, which is why I get annoyed having to stop and carry out pointless tasks like this. I want to update in one go without having to constantly go through this rigamorale. I don't want to have to do this on every Linux distribution I install or encounter. I don't want to have to do this whenever I create a clean install to test something quickly. Maybe you don't have m
Now can we get benchmarks for X? (Score:2)
I keep hoping for faster and smaller....
mark "silly me"
Am I missing something here? (Score:5, Insightful)
linux-2.6.0-test5 - 992.06 - Uni
linux-2.6.0-test5 - 1017.43 - Dual
linux-2.6.0-test5 - 5406.68 - Quad
Does this mean that you only gain 3.49% when adding a 2nd processor? Obviously I don't expect things to scale linear but 3%!? Am I missing something here? And then 81.65% for quad? I'm not trolling, I'm looking for someone to explain what I'm missing.
Re:Am I missing something here? (Score:4, Informative)
Yes, the number for dual is not 1017, but more like 1545.
Here are the actual numbers for 2.6.0-test5 and the compute workload:
1 - 992.06 - 100%
2 - 1545.03 - 155%
4 - 5175.28 - 521%
Now for why the 4 processor case is actually 5 times better than the single CPU case, I do not know enough about the benchmarks to comment.
Re:Am I missing something here? (Score:2)
Oops, pot kettle black. The UP number should be more like 1017, but the point still stands.
Re:Am I missing something here? (Score:2)
Re:Good Question, Bad Arithmetic (Score:4, Informative)
Odd but not impossible.
For example, if in the single cpu config the processes are doing a lot of memory-cache missed then having 4 cpus (with 4 times more the amount cache) could reduce the number a cache misses and so could make the quad configuration more than 4 times faster.
The same reason could explain why 2 cpus are not faster than one: if 2 caches are not large enough and if the processes have a very bad locality then you may get as much cache misses with the dual cpu system than with the single cpu system.
Re:Good Question, Bad Arithmetic (Score:2, Funny)
It may seem like i was on crack, but I promise they were prescription meds.
The kernel isn't everything (Score:3, Informative)
Scaling, et al (Score:2, Interesting)
Second, unicasting looks to be slower. Ugh. I don't like that. That suggests to me that there are segments of code which are optimized for multi-processor use - which is great - but either there aren't uniprocessor versions, or the uniprocessor versions are highly non-optimal.
Third, scaling needs to be improve
Re:I just can't do it (Score:2)
Re:Some explain to me in layman terms what the hel (Score:2)
Re:Some explain to me in layman terms what the hel (Score:2, Informative)
strike one off (Score:2, Informative)
Re:who cares? (Score:2, Informative)
Re:who cares? (Score:2)
I was really waiting for a Gentoo zealot to pop out and say "all I have to do in Gentoo is type 'emerge update' and my whole excellent portage system is updated! Yay!"
Re: Debian Zealot (Score:2, Funny)
Re:who cares? (Score:2)
Linux!=user space programs
Hence, your complaints are not about Linux, but about a bunch of programs that run on many other OSs/kernels as well as Linux.
Re:who cares? (Score:4, Insightful)
Re:who cares? (Score:3, Insightful)
Everything you "want" has nothing to do with linux.
That's like asking ford to build better tires or make higher octane gas.
ASK the right people for the features you want... the people writing the software you want improvements. in...
Because you know, Microsoft Windows Stinks because I can't do live editing in Adobe Premiere, same as I wont get excited about Internet Explorer until www.theonion.com get's better graphics...
do you understand how you sound now?
Re:who cares? (Score:2)
On the contrary, everyone has the right to be ignorant. You can try to educate people, tell them Linux is just the kernel, but they don't have to listen to you. What are you going to do about it? Nothing.
Dumbass Bill of Rights:
1) Everyone has the right to be ignorant
2) Everyone has the right to get upset about other people's ignorance
3) Everyone has the right to laugh at all the people exer
Re:who cares? (Score:2)
Mandrake. Almost as easy as Windows (no automatic checking mode turned on). Redhat's up2date is pretty much the same.
Re:who cares? (Score:2)
Right on the money. I used Mandrake on client machines and RH on servers in the pre-rhn (up2date) days. Now I use just RH, and can update, remove, install, etc. just about any package, even if I am not sitting in front of that particular box.
If the update fails, it tells me and I can do something about it. I can easily make any package where you can't update it or uninstall it unless yo
Re:who cares? (Score:2)
What does Mozilla have to do with a kernel?
2) Make Open Office slicker/ handle Mickeysoft documents better.
Again what does an office app have to do with the kernel?
3) Make a spreadsheet that doesn't suck.
I'm sensing a pattern here.
4) Make is to that I can actually cut and paste weird stuff between applications.
That's a function of X not the kernel.
5) Make is so that my PC can get updated just by clicking on items and not chasing down library incompa
Re:who cares? (Score:2)
Hey, if you wanted to say "Java", just say it. Otherwise, it's there.
2) Make Open Office slicker/ handle Mickeysoft documents better.
Constantly being worked on.
3) Make a spreadsheet that doesn't suck.
Gnumeric?
4) Make is to that I can actually cut and paste weird stuff between applications.
Ouch!
5) Make is so that my PC can get updated just by clicking on items and not chasing down library incompatiblites or typing "rpm --force" or "make i
Re:who cares? (Score:2)
Yeah!
2) Make Open Office slicker/ handle Mickeysoft documents better.
I haven't had any problems with the 1.1 series. I like it a lot better than MS-Word/MS-Excel/Powerpoint, and the import has been pretty much flawless.
3) Make a spreadsheet that doesn't suck.
There are spreadsheets that don't suck?
4) Make is to that I can actually cut and paste weird stuff between applications.
Although not universal, I've not had a problem with cutting/pasting