Benchmarks For Ubuntu vs. OpenSolaris vs. FreeBSD 131
Ashmash writes "After their Mac OS X versus Ubuntu benchmarks earlier this month, Phoronix.com has now carried out a performance comparison between Ubuntu 8.10, OpenSolaris 2008.11 and FreeBSD 7.1. They used a dual quad-core workstation with the Phoronix Test Suite to run primarily Java, disk, and computational benchmarks. The 64-bit build of Ubuntu 8.10 was the fastest overall, but FreeBSD and OpenSolaris were first in other areas."
Glad to see FreeBSD getting some press (Score:5, Informative)
I was a bit disappointed by the results, being a FreeBSD fan myself. However, in my quick scan of the article, I didn't see any mention of how they configured the OS. If they truly used the stock 7.1-BETA2 install, that would mean that debugging mode is enabled in the kernel (and maybe the userland, I'm not 100% sure here). Unless I've misunderstood FreeBSD's release methods over the years, they don't disable the debugging until either the RC builds or maybe even the final release tag.
Still, FreeBSD came out on top on 3 of the tests -- not bad for a beta release. I can't wait for 7.1, as using 7.0 on my desktop since its release has been great. I just hope the fully-virtualized IP stack within jails made it into 7.1, as well as a slightly more stable ZFS.
Re:Ubuntu may be fast... (Score:2, Informative)
.. but what it's missing is the ability to easily uninstall it. It's not the only distro not to be easily uninstallable
sudo rm -rf /
Re:Right. (Score:5, Informative)
Except that they tested FreeBSD 7.1 beta 2. FreeBSD betas are compiled with extra debugging and checking code which slows the end result down a lot. This includes the WITNESS kernel flags and the malloc checking. These encourage early and reproducible failure for bugs. For release versions of FreeBSD, these are turned off, which generally gives a noticeable speed increase. I note that the Solaris version they tested was a release candidate too, so I wonder if the same is true there.
A lot of their benchmarks seemed to be CPU-limited, with little OS involvement (e.g. FFT, RSA). Differences here are likely to be more down to malloc() implementation than anything in the kernel. In a FreeBSD beta, malloc will be adding guard pages and initialising data to a known value to check for overflows. In Solaris, I'm not sure what the current malloc() strategy is - last time I used Solaris it was still using a brk()-based malloc() (where FreeBSD and Linux both now tend to use mmap()-based versions).
Re:Ubuntu may be fast... (Score:5, Informative)
Uninstalling FreeBSD means deleting the partition. It either uses the boot loader you already had installed, or it installs a multiboot menu that fits in the MBR, so continues to work when the partition has gone away. If you install Ubuntu, I believe it installs grub and points the MBR at stuff on your /boot partition. If you destroy this, you will not be able to boot any OS.
Not that this is a major problem, since uninstalling an OS (outside of a VM) is not something that many people do very often.
Re:Glad to see FreeBSD getting some press (Score:5, Informative)
debugging mode is enabled in the kernel (and maybe the userland, I'm not 100% sure here).
Betas generally have a few malloc flags set, which make malloc() a fair bit slower, resulting in everything in the userland being slow. The point of a beta is to catch bugs before the shipping release, so everything is run in debug mode.
I'm also looking forward to 7.1, although I'm sad that the per-vchan volume control patches appear to only be in the 8.x tree. These implement the OSS 4.x ioctls and allow applications to just open /dev/dsp and write sound there, with simple ioctls to set the volume. 7.0 supports the ioctls, but they set the master volume, not the virtual channel's volume.
Re:What about the Sun Studio compiler? (Score:4, Informative)
While one could argue that the compiler is part of the OS it's indeed replaceable so I would had prefered if they had used the same version of GCC and not different for each OS.
I'm a huge FreeBSD fan. However, I don't have a problem with them testing the compiler as it was shipped with the OS because it's the one officially supported. Since that's the compiler that 99.9% of FreeBSD users will have, that seems like a reasonably fair baseline comparison.
Similarly, they explicitly state that "[a]side from changes made by the Phoronix Test Suite (and adding the GNOME packages to FreeBSD), all operating systems were left in their default configuration." I'm sure all of them could be tuned for higher performance on this benchmark, but I think out-of-the-box numbers are valuable.
Re:Right. (Score:5, Informative)
Except that they tested FreeBSD 7.1 beta 2. FreeBSD betas are compiled with extra debugging and checking code which slows the end result down a lot. This includes the WITNESS kernel flags and the malloc checking.
This is only true in HEAD which will eventually be FreeBSD 8.x. They are turned off in the FreeBSD 7.x branch (RELENG_7)
Solaris Requires tuning (Score:5, Informative)
I have not played with Open Solaris but with normal Solaris you need to set parameters in the /etc/system file to get good performance. By default Solaris is set very conservative. In many tests I have run Solaris may not be the fastest with single test but under a heavy load with many applications running my experience has been it can handle a much bigger load then Linux on the same hardware. I use both but for backend heavy loaded servers I would choose Solaris.
Re:Ubuntu may be fast... (Score:3, Informative)
They all require tuning (duh) (Score:4, Informative)
All three come with tunable performance parameters. All three can have their performance boosted even further by recompiling everything optimized for the particular hardware being used, possibly using specialized compilers (e.g. from Sun or Intel). But that's not the point, IMO. This isn't (or shouldn't be) a pissing match--this should be an opportunity to improve all three systems by seeing where their strengths and weaknesses are, and working to bolster their weaknesses and improve their strengths.
In my experience, these sorts of tests on free/libre/open-source systems quickly become out-of-date because the developers take them as a challenge, and that's a good thing for everyone! :)
Ff your tests were more than a couple of years ago, they're probably so out-of-date as to be utterly meaningless, but that's a separate issue. Personally, I'm a big fan of all three systems and want to see all three thrive and grow and improve. This kind of testing can only help with that, once you get past all the dick-waving by narrow-minded advocates.
Re:Ubuntu may be fast... (Score:3, Informative)
the windows boot loader will not recognize a linux install
My notebook, which has Windows XP and Ubuntu Linux, and which uses the Windows bootloader to boot the Linux partition, disagrees with you.