FreeBSD 8.0 vs. Ubuntu 9.10 Benchmarks 268
An anonymous reader writes "Phoronix has brought benchmarks comparing the FreeBSD 8.0-RC and Ubuntu 9.10 Alpha 6 operating systems. FreeBSD rather ends up taking a wallop to Ubuntu Linux, but there are a few areas where FreeBSD 8 ran well. They also posted benchmarks comparing this near-final FreeBSD 8.0 build to that of FreeBSD 7.2 to show performance improvements there but with a few regressions."
What's the point. (Score:5, Insightful)
Summary (Score:5, Insightful)
The article is of little use other than tell the general populace of Slashdot that FreeBSD 8.0 and Ubuntu 9.10 are right around the corner, and that we should be hyped. Also FreeBSD 8 is a little faster than FreeBSD 7.2 but a lot slower than Ubuntu Linux 9.10
I'm not surprised, however I do belong to the group that does not really care about relative performance to other OS's as performance is only one of the aspects from the vector of decisions we had to make to finally choose FreeBSD for mass-scale deployment.
Benchmarks... (Score:3, Insightful)
Benchmarks are useless. There are way more important things to judge an operating system then "speed".
Does Ubuntu have nearly as good of documentation? No. It has that "info" nonsense.
Does Ubuntu provide a stable platform to build a server? No. It, like most linux distros, changes whole versions during updates. That isn't stable.
Does Ubuntu provide a way to strip itself down to the bare metal? Ain't as easy as the BSD's.
Is Ubuntu built around solid engineering and design, or politics? Depends--Ubuntu seems to be less afraid of the big bad FSF as other distros, but it still is steeped in an OS built for politics. FreeBSD is pretty tame and tends to focus on solid engineering rather than political maneuvering.
But really, Comparing FreeBSD to Ubuntu is like comparing OpenSolaris to Windows 7. FreeBSD is largely a server operating system were as Ubuntu is an end user operating system. And if you are comparing server operating systems, there are far more important criteria than "speed". Things like version stability are vastly more important.
Re:What's the point. (Score:5, Insightful)
Just a plug for the Phoronix Test Suite?
Awful (Score:1, Insightful)
lrn2preposition
Truly crap-tastic charts (Score:5, Insightful)
Eight pages of bar charts, each gray-on-gray. On half of them, shorter bars mean better performance, on the other half, longer is better; the only way to know which is which is in a legend, written in a small font.
Here's a suggestion: color-code the bars! Green is good, red is bad, yellow is in the middle of the road. For bonus points, choose the saturation based on magnitude of the differences. If the numbers are close, go with grayer bars, if the differences are dramatic, use dramatic colors.
Finally, how about a line chart at the end showing all of the numbers in one place? Yeah, you'd need to convert everything to be consistent if longer or shorter is better, but that's a good idea anyway.
Re:Benchmarks... (Score:3, Insightful)
1. Agreed, I think neither is really better and use depends on what you need, which of course some Linux and BSD zealots seem to disagree on - apparently the main free *nix are now entrenched enough to be part of the unix holy wars.
2. Varies, FreeBSD doesn't have perfect docs either and other distros do have better docs
3. Lrn2LTR
4. Yes, it does, there's a 10MB barebones installer for the dedicated and if you need less, pick another distro, even FreeBSD will probably be tons of tweaking at this point.
5. Political strawman yay
6. Call the whambulance.
Re:Benchmarks... (Score:3, Insightful)
Re:Benchmarks... (Score:3, Insightful)
Hah, I see what you did there. You ripped on Ubuntu for being steeped in politics, while ignoring the available facts, i.e. that Ubuntu 10.4 is a stable Long Term Support release, and that it wipes the floor with FreeBSD 8.0 in performance.
That's pretty funny. I mean, if you did it on purpose.
Re:Summary (Score:3, Insightful)
Not disputing the conclusions of the memory, I/O benchmakrs, part of what they benchmarked is the compiler — FreeBSD's gcc-4.2.1 vs. Ubuntu's 4.4.1. I'm not surprised, GCC's newer release both compiles faster and produces faster binaries.
You could say, FreeBSD is at fault (and thus deserves bad rep) for including an outdated compiler, but, on the other hand, FreeBSD's choice may prove wiser, when bugs in the hot new compiler surface... The compiler's history — as, really, that of any sufficiently complex piece of software — teaches, it is wise to stay several releases behind. Bugs in the older releases are known and all platforms — certainly including FreeBSD — merge fixes into their repositories.
Their choice of using ImageMagick as a test is particularly suspect — that software has so many options (which graphics back-ends to include, whether to use OpenMP, etc.) and varies so greatly between its own frequent minor releases, that I'm sure they built it with subtle (but timing-affecting) differences between platforms... For just one example, FreeBSD's port of ImageMagick [freshports.org] runs all of their bundled self-tests after compilation by default, which takes quite a while. Unfortunately, the testers don't even mention, how exactly they built the stuff. If they used the port on FreeBSD, did they change any options? If they did not use the port, then they didn't build ImageMagick the way the users will be building it... And if they did use the port and flipped some features, did they ensure an identical match between two FreeBSD versions and Ubuntu?
Re:benchmark fail! (Score:1, Insightful)
Re:Sigh... (Score:4, Insightful)
You can only do an equally good job with MacOS.
For this grant priveledge you get a meagre selection of hardware that will
either gouge you or leave you wanting. You also get an alien environment
with a number of annoying quirks and inferior package management to any
Unix. Package managment is a "Unix thing" and not just a Linux thing. This
is one area where MacOS demonstrates it's not really Unix.
Some of the proprietary tools you get with MacOS might be moderately more
useful but they will have quirks of their own, suffer from NIH syndrome
and may also suffer from addressing problems in a superficial manner.
BSD is at least a proper Unix.
Re:What's the point. (Score:5, Insightful)
I must say that I'm really fed up with these sorts of benchmarks. They insult the intelligence of IT/technical people AND business people/endusers.
I need to find or create a site that actually benchmarks a variety of systems (windows/linux/unix/bsd) for ACTUAL scenarios that the users of the systems expect them to perform. Not sure how important LAME encoding is for BSD people - I know I don't use it for that. Not sure how many Ubuntu users run DB servers or firewall routers.
Can anyone point me to someplace like this? Someplace where I can see, for example, DB driven dynamic pages served per second - concurrent DB users, scalability to 4+ processors, etc?
I want metrics like SAN performance, DB performance, mail server performance, HA, clustering, firewall performance etc for server "distros" (Windows Server, AIX, Solaris, BSD, Linux, OSX Server)
I want metrics like graphics benchmarks, app response time, those classic GUI task tests for desktop "distros" (Windows, OSX, Linux, BSD).
It annoys me that I see benchmarks that are so narrow and pointless. I can't make reasonable non-fanboy type comparisons when the benchmark doesn't include anything that matters to me.
Re:Ubuntu *is* Linux for a lot of people (Score:3, Insightful)
This should be interesting... Someone who doesn't understand why everyone doesn't think alike.
So we all know how qualified you are when it comes to unbuntu. I use 8.04 since it's the latest LTS version and I haven't had any problems with it. Your mileage may vary.
Well maybe it would have helped to press that button label "Unlock" and then type your password.
Again it would have gone a whole lot easier if you just clicked the unlock button. Quit being a drama queen.
Whew... I was on pins and needles wondering if you would find a Linux distribution that you liked. *sarcasm*
Maybe because people have different tastes? A much better question you should have asked is "If everyone else thinks Ubuntu satisfies their needs, what am I doing wrong?" or the best question to ask is "I wonder if I should ask for help since I seem to be in the minority of people who think Ubuntu is too difficult to use?"
I heard some valid criticisms about Ubuntu, but not being user friendly isn't one of them.
I thought people who get caught up in the Windows vs. Linux vs. OS X vs. BSD were pretty small minded. How small a mind do you have to have to fight over which distribution of a single OS is the best? Heaven forbid someone else uses something different then you.
My apologies for making these snide remarks, I'll be in a much better mood once my coffee kicks in. Anyway, what kind of response did you expect?
Maybe you should have reworded your comment into:
I wonder if they will post a benchmark comparing FreeBSD 8.0 to my favorite linux distro?
Im colourblind you insensitive clod (Score:2, Insightful)
Re:What's the point. (Score:2, Insightful)
True. And let's also see some benchmarks with the system under heavy load. Run it as a webserver with thousands of page requests a second. Run it as a database server under a realistic load (not simply inserting a few records).
I could give a rats ass about how fast it compiles some app, show me some REAL benchmark!
Debugging/self-check in the FreeBSD kernel (Score:3, Insightful)
People mentioned the self-checks and debugging features that used to be turned on in FreeBSD development branches and beta releases.
Self-checks, which are the major source of kernel slowdowns in those kernel options, are not turned on in the 8.0 release candidates.
Debugging is on, but unless you are very short of memory it should not cause a noticeable slowdown.
FreeBSD's slowness in these benchmarks can be attributed to two factors:
1) the compiler. The GPL v3 is unacceptable for FreeBSD, so newest GCCs cannot be used as the base compiler. Users can choose to install a new gcc on their own (as a port) and then even go and compile all ports and even the base system with that new compiler (some parts might not have been cleaned up to comply with new language strictness that might have come with new gccs).
2) threading, as in the userland threading library support. It is very hard to tell whether there is some performance problem in FreeBSD's thread libraries or whether applications just happen to have been optimized and tested only with Linux.
That happens a lot and you can also see Solaris with it's M:N threading fail miserably for some threading benchmarks that do dump things, such as just creating and destroying threads at a high rate.
%%
The problem of "thread benchmarks" benchmarking bogus things and/or just having been written with Linux' thread model in mind has been going on for 12 something years now. Benchmarking thread systems in a realistic manner is very difficult. In real-world applications you don't create and destroy threads at a high rate and you minimize locking. Benchmarking this is almost as hard as benchmarking programming languages.
I haven't benchmarked threading libraries, knowing that it will take time that I don't have right now. I can tell you, however, that just the I/O subsystem in FreeBSD, as in filesystems and networking, isn't any slower than Linux. Not to mention GbE and today's disks are too slow to really show an OS difference for most tests anyway.
%%
The real question of I/O subsystems will come in when ZFS+Zraid is standard in FreeBSD and BTRFS is standard in Linux. In a couple of years from now nobody will understand why we ran today with no snapshots, with the raid hole (from block layer raid systems) and without transparent compression in the subtrees where we want it.
But these filesystems are complicated and there's some real performance difference visible.
%%
An area completely overlooked in the benchmark is the VM subsystem. Namely - what happens when you overload your RAM and paging commences? Linux used to make very bad choices here (dropping readonly pages, which is a wise thing as such, at rates about 10 times higher than wise).
FreeBSD used to be the go-to OS on memory shortage thanks to John Dyson's VM work (backed by a large database company that provided support and a realistic benchmarking environment during that development).
But today? Nobody knows. I'm not aware of any benchmarks that you can download that simulate memory stress and map the tradeoffs that the OS makes.
In general, the biggest obstacle to improve Linux, FreeBSD and everybody's else's OS performance is a lack of high-quality benchmarks.
Why don't people develop more benchmarks? Because they get annoyed that today, in 2009, no realistic OS benchmark will show a single number as a result. All OS benchmarks today can only make a map of what tradeoffs the OS chose, what part of the running application suite got the short end in favor of what other part. This isn't sexy and publishers don't like it.
People like reality reduced to single numbers, but in the area os OS benchmarks (and language benchmarks) that party is over.
Myself, I found myself gasping many years ago when I benchmarked network I/O versus userland CPU load. I hammered a couple of GbE interfaces while at the same time running moderate memory-intensive CPU benchmarks (with no network access from those CPU lo
Re:Benchmarks... (Score:3, Insightful)
Its important to those of us that know by not including extra stuff on the machine, that extra stuff can not possibly be exploited, cause problems, waste resources or any number of other reasons that don't come to mind right at the moment. Including extra stuff is almost always a bad idea when you're doing a server. Desktops are a little different. Interestingly enough, Ubuntu is desktop focused with some server on the side. FreeBSD is server only, with some people who refuse to give up on making it a desktop OS, which they actually do a reasonably good job at if you're technical enough to deal with some of the quirks along the way.
You are of the same mentality as MS. Disk space and memory is cheap, who cares if we waste space with unneeded stuff!
Thats fine for a desktop OS where making things work off the bat for the user is more important than being intelligent. Personally, I'd rather you give me a ton of options and sane defaults rather than an installer that basically just copies everything from the cd/dvd/nfs mount to the drive and reboots.
However, its certainly doable with Ubuntu, it just takes more work than with FBSD. The difference is really in the installer. Ubuntu is far more ignorant-user-friendly than FreeBSD, which is perfectly acceptable as its aimed more at being easier to use. I've been using FBSD since 2.2.x and with the exception of a minor change allowing for selection of your X desktop environment the FBSD installer hasn't changed since then and I still made a mistake and wasted an install during an install of an 8 preview. Entirely my fault for not paying attention, but its one of those things that the Ubuntu installer most certainly would have red flagged or just not allowed at all. Different designs for different targets, both are better than the other in the areas they intend to be.