Slashdot Log In
Benchmarks For Ubuntu vs. OpenSolaris vs. FreeBSD
Posted by
timothy
on Tue Nov 25, 2008 10:53 AM
from the three-way dept.
from the three-way dept.
Ashmash writes "After their Mac OS X versus Ubuntu benchmarks earlier this month, Phoronix.com has now carried out a performance comparison between Ubuntu 8.10, OpenSolaris 2008.11 and FreeBSD 7.1. They used a dual quad-core workstation with the Phoronix Test Suite to run primarily Java, disk, and computational benchmarks. The 64-bit build of Ubuntu 8.10 was the fastest overall, but FreeBSD and OpenSolaris were first in other areas."
Related Stories
[+]
Technology: Ubuntu 8.10 vs. Mac OS X 10.5.5 Benchmarks 328 comments
An anonymous reader writes "As a sequel to their Is Ubuntu Getting Slower? Phoronix now has out an article that compares the performance of Ubuntu 8.10 to Apple's Mac OS X 10.5.5. They tested both the x86 and x86_64 spins of Ubuntu and threw at both operating systems a number of graphics, disk, computational, and Java benchmarks, among others. With the Mac Mini used in some of the comparisons, 'Leopard' was faster, while in others it was a tight battle."
Submission: Ubuntu vs. OpenSolaris vs. FreeBSD Benchmarks by Anonymous Coward
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
What about the Sun Studio compiler? (Score:5, Insightful)
Various versions of GCC. While one could argue that the compiler is part of the OS it's indeed replaceable so I would had prefered if they had used the same version of GCC and not different for each OS.
It would had been very interesting to see the Solaris results using Sun Studios CC as well (I think it's also available for Linux nowadays?)
Re:What about the Sun Studio compiler? (Score:4, Informative)
While one could argue that the compiler is part of the OS it's indeed replaceable so I would had prefered if they had used the same version of GCC and not different for each OS.
I'm a huge FreeBSD fan. However, I don't have a problem with them testing the compiler as it was shipped with the OS because it's the one officially supported. Since that's the compiler that 99.9% of FreeBSD users will have, that seems like a reasonably fair baseline comparison.
Similarly, they explicitly state that "[a]side from changes made by the Phoronix Test Suite (and adding the GNOME packages to FreeBSD), all operating systems were left in their default configuration." I'm sure all of them could be tuned for higher performance on this benchmark, but I think out-of-the-box numbers are valuable.
Parent
Re: (Score:2)
At least earlier I think you could switch in rc.conf or similar if you wanted to use gcc 3.x or 2.95.x for instance, both may have been available, I don't remember. I don't think that is much of tuning, not much more than trying to benchmark KDE by different versions, or even various versions of Windows on different laptops.
If it's about the compiler version (or options) I know I can switch compiler and get a better result, if it's some other tweaking or totally different kernel or system libs or such it ma
Re: (Score:2)
Re: (Score:2)
That's why Sun cc would had been fun in case it had owned gcc =P
Re: (Score:2)
Glad to see FreeBSD getting some press (Score:5, Informative)
I was a bit disappointed by the results, being a FreeBSD fan myself. However, in my quick scan of the article, I didn't see any mention of how they configured the OS. If they truly used the stock 7.1-BETA2 install, that would mean that debugging mode is enabled in the kernel (and maybe the userland, I'm not 100% sure here). Unless I've misunderstood FreeBSD's release methods over the years, they don't disable the debugging until either the RC builds or maybe even the final release tag.
Still, FreeBSD came out on top on 3 of the tests -- not bad for a beta release. I can't wait for 7.1, as using 7.0 on my desktop since its release has been great. I just hope the fully-virtualized IP stack within jails made it into 7.1, as well as a slightly more stable ZFS.
Re:Glad to see FreeBSD getting some press (Score:5, Informative)
debugging mode is enabled in the kernel (and maybe the userland, I'm not 100% sure here).
Betas generally have a few malloc flags set, which make malloc() a fair bit slower, resulting in everything in the userland being slow. The point of a beta is to catch bugs before the shipping release, so everything is run in debug mode.
I'm also looking forward to 7.1, although I'm sad that the per-vchan volume control patches appear to only be in the 8.x tree. These implement the OSS 4.x ioctls and allow applications to just open /dev/dsp and write sound there, with simple ioctls to set the volume. 7.0 supports the ioctls, but they set the master volume, not the virtual channel's volume.
Parent
Re: (Score:2)
I just hope the fully-virtualized IP stack within jails made it into 7.1, as well as a slightly more stable ZFS.
No virtual IP stack yet. ZFS is slightly more stable, but still has a ways to go.
Ubuntu performance (Score:4, Interesting)
The reason that I never really seriously used Linux on my PC laptop was that Ubuntu was sluggish, even with the newest ATI drivers, compared to Windows. Maybe people have good experience with nVidia drivers there, but Windows is a lot more usable as a desktop for me on the performance side of things. Granted, my main computer is a MacBook Pro running Leopard, but I can't imagine putting Linux back on my old PC laptop for when I need to use it.
Re: (Score:3, Insightful)
These days a full install of Ubuntu, with all it's bells and whistles is going to be slower than a bare install of XP. The nice thing is that you don't need all that cruft, and it's pretty easy to install a command line system and add things as needed. Use a lightweight WM or desktop like fluxbox or XFCE and you're set.
A particular distribution vs different OSs (Score:2)
What about some combined loads? (Score:5, Interesting)
Interesting results, and great if you're planning a server, but what about desktop use?
How well does each OS do when doing something like playing back audio/video, and handling background processing loads? What about performance and system response as the load climbs up? (load averages of 5/10/20 ?).
Only because I've seen Linux systems start to crumble around 5 (uniproc machine), and easily get unusuable, but have heard reports of BSD machines being able to still play MP3s without skipping/suttering even around 20 or so...
(And yes, I'll allow tweaking system priorities - it only gets you so far, and impacts the other background processing tasks, to which we'll also be interested in how long they take to run. So renicing the media player to -20 works, but not if it makes all the other tasks take 10x as long to finish...).
Re: (Score:2)
Let's bring BeOS in on that test :)
You could IM and browse the web on that OS without making your MP3s skip, all on a Pentium 133 w/ 64MB ram. None of its contemporaries were even close in terms of UI responsiveness under load and smooth media playback, and no bloated modern OS is even close. Damn impressive.
Re: (Score:2)
How do you measure interactive performance? When you're just crunching numbers, or doing sustained reads from a disk it's pretty easy to get meaningful numbers. But when you have 5 or 6 different things going on at the same time, it's hard to replicate those conditions exactly on different systems. And if you do, what sort of metric are you going to use that will look good on a graph and actually tell you something about how well the system is performing?
In your face Ballmer! (Score:2)
Sorry, I got a bit carried away there. Eh, what was the article about again?
Solaris Requires tuning (Score:5, Informative)
I have not played with Open Solaris but with normal Solaris you need to set parameters in the /etc/system file to get good performance. By default Solaris is set very conservative. In many tests I have run Solaris may not be the fastest with single test but under a heavy load with many applications running my experience has been it can handle a much bigger load then Linux on the same hardware. I use both but for backend heavy loaded servers I would choose Solaris.
Re: (Score:2)
I have not played with Open solaris but ubuntu was running on ext2 with linux if you want performance you need to pick an alternative file system (i find reiserfs to be good but support for it seams dwindling and answers such as use a supported fs (despite reiserfs being supported) are not uncommon), anyway my point is that these tests are pointless!
Re: (Score:3, Funny)
They all require tuning (duh) (Score:4, Informative)
All three come with tunable performance parameters. All three can have their performance boosted even further by recompiling everything optimized for the particular hardware being used, possibly using specialized compilers (e.g. from Sun or Intel). But that's not the point, IMO. This isn't (or shouldn't be) a pissing match--this should be an opportunity to improve all three systems by seeing where their strengths and weaknesses are, and working to bolster their weaknesses and improve their strengths.
In my experience, these sorts of tests on free/libre/open-source systems quickly become out-of-date because the developers take them as a challenge, and that's a good thing for everyone! :)
Ff your tests were more than a couple of years ago, they're probably so out-of-date as to be utterly meaningless, but that's a separate issue. Personally, I'm a big fan of all three systems and want to see all three thrive and grow and improve. This kind of testing can only help with that, once you get past all the dick-waving by narrow-minded advocates.
Parent
Phoronix Test Suite (Score:2)
Does anyone know if the Phoronix Test Suite will work under OpenBSD and NetBSD too? Says on the website: "Runs On Linux, OpenSolaris, Mac OS X, & FreeBSD Operating Systems"
I'm curious as to how the other BSDs would perform.
They should include Vista (Score:2)
I am curious to how it actually performs and not just what most the slashdotters say.
It may actually suck and I am curious as to how much.
I am contemplating leaving vista where I do php, apache, and java development. I wonder if there is an advantage at all.
Mostly pointless (Score:3, Insightful)
Except for Bonnie++, all of their benchmarks are compute-bound. In other words, they're benchmarking the bundled compiler, not the distribution.
The one exception is Bonnie++, on page 6, which measures raw filesystem performance... and is something that is known to greatly depend on how old and how full a given filesystem is.
Re: (Score:2)
Is that true even if you install grub/lilo to MBR instead of the bootable partition? Do it need anything from /boot? I doubt that?
Sure the boot loader wil remain, but so what? Also it's quite obvious that you need to update it so ..
It's not like Windows offers a way to easily uninstall it either, or even ask before overwriting it ..
Re: (Score:2)
Grub needs it's stage files and config file. Lilo however, lives entirely in MBR.
Re: (Score:2)
You have to do what now?
Ubuntu uses Grub for choosing what to boot and that has the active flag. No matter what OS you use, if you want something else to handle booting you have to use fdisk or the like for changing the boot flag.
Re:Ubuntu may be fast... (Score:5, Funny)
Yes, I, too, find this nut and bolt set inadequate for the purpose of assisting Chinese Rhinoceros to learn Western astrology. It seems silly that purple monkey dishwasher.
Parent
Re: (Score:2)
Try to uninstall windows the same way... you always have to mess with some boot loader, or at least, whatever that replaces your boot loader.
There are a lot of space to complain about linux missing something or making something harder (playing some specific games, not having some windows-only program, not supporting some hardware that developed/documented dri
Re:Ubuntu may be fast... (Score:5, Informative)
Uninstalling FreeBSD means deleting the partition. It either uses the boot loader you already had installed, or it installs a multiboot menu that fits in the MBR, so continues to work when the partition has gone away. If you install Ubuntu, I believe it installs grub and points the MBR at stuff on your /boot partition. If you destroy this, you will not be able to boot any OS.
Not that this is a major problem, since uninstalling an OS (outside of a VM) is not something that many people do very often.
Parent
Re: (Score:3, Informative)
Re: (Score:2)
To be fair, if you decided to use NTLDR instead of GRUB and you delete your Windows partition, the system is rendered unbootable as well. So in this instance, Ubuntu is no worse (or better) than Windows.
FreeBSD does do an excellent job by making the bootloader small enough to fit in the MBR.
Re: (Score:2, Funny)
thats not how it works.
ubuntu software runs on the windows that pcs run on and so you need to remove the ubuntu layer to get to the normal windows again.
when you uninstall ubuntu it puts the desktop colours back to the normal windows ones so you can run your programs again. though you need to reinstall some programs still.
if windows was not also installed there is no way to run ubuntu as you still need to move the mouse.
Re: (Score:2, Informative)
.. but what it's missing is the ability to easily uninstall it. It's not the only distro not to be easily uninstallable
sudo rm -rf /
Re:Ubuntu may be fast... (Score:4, Insightful)
Parent
Re: (Score:2)
Huh? Ubuntu is an OS not an application. It installs its own boot loader called grub. It does this because the windows boot loader will not recognize a linux install. Perhaps your beef, if you truly have one, is with MS. They could make their boot loader play nice with other OS's. I believe you can load windows with grub just fine. The /mbr restore for windows is to wipe grub and go back to using the windows loader.
If you think any other non-MS OS is easier to "uninstall" then youre just wrong.
Re: (Score:3, Informative)
the windows boot loader will not recognize a linux install
My notebook, which has Windows XP and Ubuntu Linux, and which uses the Windows bootloader to boot the Linux partition, disagrees with you.
Re: (Score:2)
Forget uninstall. Is there even yet a way to *install* Windows without trashing the MBR w.r.t other OSes? If not, I say Windows is not ready for the desktop.
Re: (Score:3, Insightful)
Even with all that extra 'default' weight Ubuntu still shines on except when running, eh, Java.
From my personal, and non-scientific experience, I've found OpenSolaris to have more 'default' weight than Ubuntu.
Re: (Score:2)
Yeah, because Solaris and FreeBSD are so stripped and optimized for the purpose by default? =P
I still think it's somewhat useless since they had various versions of the compiler, you can't draw any conclusion about the kernel/OS speed when the compiler varies and you can't draw any conclusions on the compiler when the kernel/OS differ. So ...
Put two rowers in whatever boats they happen to have, let them row for a goal, see who wins and decide that person was the best rower? Or which boat was the best one ..
Re: (Score:2)
I still think it's somewhat useless since they had various versions of the compiler, you can't draw any conclusion about the kernel/OS speed when the compiler varies and you can't draw any conclusions on the compiler when the kernel/OS differ. So ...
I disagree. While these benchmarks are indeed of limited complexity and utility, they offer a snapshot of out of the box performance on 3 OSes. Obviously you can optimize all of them substantially, try bleeding edge kernels, later revisions, etc.
Re: (Score:2)
I know they tell how the system performs in its default state, but in that case why even bother to compile the benchmarks? Shouldn't you be using whatever versions is available as packages if there is any?
And default versions doesn't matter much, if anyone is serious enough to start compiling all their own applications they sure will be able to switch compiler version, and there may be both advantages and disadvantages with a new or old version so it all depends on what you want. In this case Ubuntu took ma
Re:Right. (Score:4, Insightful)
Parent
Re: (Score:2)
I absolutely agree with you.
Re:Right. (Score:5, Informative)
Except that they tested FreeBSD 7.1 beta 2. FreeBSD betas are compiled with extra debugging and checking code which slows the end result down a lot. This includes the WITNESS kernel flags and the malloc checking. These encourage early and reproducible failure for bugs. For release versions of FreeBSD, these are turned off, which generally gives a noticeable speed increase. I note that the Solaris version they tested was a release candidate too, so I wonder if the same is true there.
A lot of their benchmarks seemed to be CPU-limited, with little OS involvement (e.g. FFT, RSA). Differences here are likely to be more down to malloc() implementation than anything in the kernel. In a FreeBSD beta, malloc will be adding guard pages and initialising data to a known value to check for overflows. In Solaris, I'm not sure what the current malloc() strategy is - last time I used Solaris it was still using a brk()-based malloc() (where FreeBSD and Linux both now tend to use mmap()-based versions).
Parent
Re:Right. (Score:5, Informative)
Except that they tested FreeBSD 7.1 beta 2. FreeBSD betas are compiled with extra debugging and checking code which slows the end result down a lot. This includes the WITNESS kernel flags and the malloc checking.
This is only true in HEAD which will eventually be FreeBSD 8.x. They are turned off in the FreeBSD 7.x branch (RELENG_7)
Parent
Re: (Score:2)
Beyond that, how hard do these tests push SMP systems? For example, the Gzip results could be explained away by compiler version differences. What would happen if it were run twice in parallel? 100 times? This test suite compares that for all I know, but I didn't see that mentioned anywhere.
Re:Right. (Score:4, Insightful)
It's kind of crazy how so many benchmark reviews completely overlook actual use and go for one or two "bullet list" type qualifiers for their benchmarks. Granted, I understand this is mainly in the interest of page hits and ad revenue, and by making it controversial they increase those things, but c'mon. Benchmarks are supposed to be pragmatic, and in order to be pragmatic, they have to operate at or near userland conditions, considering CPU, bus, memory and network speed, and the like - as they pertain to the user (whether the user is a hosting company or a desktop end user).
It seems like a pretty trivial matter to do something like this. Say, use something like MySQL for starters - it's available for a dozen or so systems (major Linux distros, OS X, Windows, etc.) It's also typically offered by the vendor, so you'd be able to get an 'ideal' setup for each release.
Or, how about something like a "Firefox benchmark" as that's user-applicable and can use all hardware. Time how long it takes to start FF on all systems with, say, 50 tabs running.
Or how about a straight-up media playing benchmark for 2D performance? Launch a dozen or so DivX videos at once and see how well it performs: measure CPU load, memory use, and the time it takes for FF to start up completely.
Or how about lengthy disk access (maybe crawling a storage tree or such) and measure the time it takes, as well as the amount of memory which gets cached for the process?
This benchmark, as well as most others, seem pretty trivial and useless, and not all that well thought out. They're certainly not scientific!
Parent
Re: (Score:3, Interesting)
Re: (Score:3, Insightful)
Linux is a microkernel? Mach is monolithic? Since when?
Re:Kernel Architecture would offer same prediction (Score:4, Interesting)
Linux is a microkernel? Mach is monolithic? Since when?
Should read, "Linux with its non-microkernel heritage"
The point was that Linux has no traditional microkernel alignment in contrast with OS X that keeps the traditions of a microkernel that when paired with a monolithic BSD interface kills a lot of the concept of what the MACH kernel was intended to do.
Originally MACH was a microkernel concept, but in its current incarnations, like OpenBSD, OS X, etc, it is no longer a microkernel by any set of definitions other than being another abstraction layer for the upper level kernel API sets.
MACH when paired with BSD, a monolithic kernel API you lose a lot of the direct hardware one request concept of a microkernel, especially on today's architectures.
Linux was true to itself in that it never attempted to abstract hardware and instead set its own rules for what was expected of the hardware, and when running on hardware that cannot meet the needs, the functionality that Linux requires must be simulated on that hardware.
So you have Linux that will outperform OS X because of its all in one nature that doesn't have to cross call API layers for kernel processes. On the other hand you have a BSD/MACH concepts like OS X that can do well for hard crunching simple tasks that funnel all the way to the MACH kernel, but when it gets to handling multiple requests, process communication gets sticky and multi-tasking can kill the once low level elegant level of performance offered.
NT has neither of these pitfalls. It has a very fast process creation system, a low level HAL, and multi-layered kernel API sets. Not only do you get the near speed of a microkernel, but you also get the robust API sets that STILL reside in true kernel layers.
On Windows this is taken to such an extreme that even Win32, which is an OS subsystem running on NT, has its own kernel32, that is technically a 'kernel' level API, yet sits all the way up in an agnostic subsystem.
There is a reason the kernel designer of MACH let it go and moved on to Microsoft and has put their knowledge and work behind NT, because they believe in the architecture, even over their own creation.
As for NT being a copy or rip off of VMS, there is some truth that the knowledge from the VMS team didn't forget what they learned when they went to Micrsoft, but also remember, they were wanting to replace VMS when at DEC even and much of their concepts where thrown by corporate politics, preventing any massive innovation to the platform that they seriously wanted to explore. This is what moved so many to go to MS so they could make the next generation OS.
NT wasn't just a overnight bastard creation, it was the best and brightest from MS and VMS and even the UNIX developers of the time...
Cutler is brilliant, but in today's kernel world, even he admits he is getting dated. (Even Windows 7 moved in a few new people to optimize in different directions reworking old standard Cutler level code in the kernel.)
So if we can say, what Culter's team did in the 1990s was more revolutionary than evolutionary, as NT really doesn't conform to VMS concepts, especially theoretical kernel concepts, then why can't we ask the OSS world today to revisit kernel architecture on a larger scale?
Instead I see articles flying around about BSD vs Linux and Linus writing about why monolithic kernel designs will alwasy be better and other experts debating that moving back to an more inclusive microkernel with modern hardware in mind would be better.
Where are the movers in the OSS word that are outside this box and why isn't the actively working on even a basic hybrid kernel technology of its own that with what kernel engineers know today will leapfrog kernel design?
Instead the big work you see on actual new kernel concepts are yet again coming from places like Microsoft Research where they are playing with singularity and other kernel concepts that range from managed code kernel designs to even frankenstei
Parent