Forgot your password?
typodupeerror
BSD Linux

FreeBSD 8.0 vs. Ubuntu 9.10 Benchmarks 268

Posted by CmdrTaco
from the two-oses-enter-two-oses-leave dept.
An anonymous reader writes "Phoronix has brought benchmarks comparing the FreeBSD 8.0-RC and Ubuntu 9.10 Alpha 6 operating systems. FreeBSD rather ends up taking a wallop to Ubuntu Linux, but there are a few areas where FreeBSD 8 ran well. They also posted benchmarks comparing this near-final FreeBSD 8.0 build to that of FreeBSD 7.2 to show performance improvements there but with a few regressions."
This discussion has been archived. No new comments can be posted.

FreeBSD 8.0 vs. Ubuntu 9.10 Benchmarks

Comments Filter:
  • by Jurily (900488) <jurily@noSpAM.gmail.com> on Monday September 28, 2009 @08:59AM (#29564481)

    ...once I'm done compiling.

    • by wall0159 (881759) on Monday September 28, 2009 @09:25AM (#29564711)
      Given that, I think it'd be fairer to compare Gentoo against Ubuntu 10.04, don't you?
    • Compiling in FreeBSD (Score:5, Interesting)

      by phorm (591458) on Monday September 28, 2009 @11:04AM (#29565869) Journal

      Actually, you'll probably do quite a bit of compiling in FreeBSD too. While you can pull binary packages, most people I know prefer to use a package manager like portsnap and compile source packages.

      If you have a script to take care of compiling the common stuff, then just "make config" everything that need user input and let it fly. Granted, most servers I've used BSD on have been dual or quad core Xeon/AMD PC's with a fair bit of RAM, so it is overall a fairly quick process, though it was still not too bad on a few P4-era celerons.

      The nice thing about FreeBSD source-compiled apps that I truly did love compared to BSD is the little tweaks you could do to avoid tons of crap dependencies. Debian used to be fairly "clean" as far as deps, but both Ubuntu and Deb are now getting quite ugly in that you get rather unwanted stuff in order to get the package you want. In most FreeBSD stuff, for example, I can check the "No X Server" box and happily compile my apps without any X support whatsoever. On Debian/Ubuntu I end up trying to install some CLI system monitoring tool or CUPS, whatever, and end up with a whackload of x.org stuff because it's tied to some font which is loosely tied to the actual package I'm trying to install. On Ubuntu desktops, trying to exorcize the demons of "Evolution" without having it remove other important stuff due to deps is near impossible. Sometimes there's a separate branch for a "cleaner" install, but often enough not.

      Of course, BSD ain't perfect. Linux tends to have a lot of "new" stuff that BSD is a little more "conservative" in bringing into the mainline (iSCSI support for example was a fairly recent addition compared to 'nix), but overall the package system is powerful indeed.

      Having not used Gentoo (yet) but knowing others who have, it seems like it might be somewhat similar in concept. The issues I've heard with are mostly in people getting to the up-and-running stage, but - similar to BSD - avoid annoying little conflicts or unwanted cruft is a lot easier than Debian/Ubuntu's precompiled binaries. Of course, I've also heard of much frustration in both if you have find halfway through a *long* compile that you missed something you should have flagged/included.

    • by bonch (38532)

      How about we see this against a version of FreeBSD that doesn't have debug on according to /usr/src/UPDATING?

      NOTE TO PEOPLE WHO THINK THAT FreeBSD 8.x IS SLOW:
      FreeBSD 8.x has many debugging features turned on, in both the kernel and userland. These features attempt to detect incorrect use of system primitives, and encourage loud failure through extra sanity checking and fail stop semantics. They also substantially impact system performance. If you want to do performance measurement, benchmarking, and optimi

  • What's the point. (Score:5, Insightful)

    by hedwards (940851) on Monday September 28, 2009 @09:01AM (#29564499)
    I'm sort of curious what the point is of comparing an alpha to a release candidate. Additionally it's a minor update versus a major update. Throwing in an older release makes it all the more pointless as I'm not seeing anywhere in the summary that they disabled debugging.
    • by bostei2008 (1441027) on Monday September 28, 2009 @09:09AM (#29564571)

      Just a plug for the Phoronix Test Suite?

      • by suso (153703) *

        Yes and we can't have advertising on the net! *sarc*

      • by WinterSolstice (223271) on Monday September 28, 2009 @11:38AM (#29566441)

        I must say that I'm really fed up with these sorts of benchmarks. They insult the intelligence of IT/technical people AND business people/endusers.

        I need to find or create a site that actually benchmarks a variety of systems (windows/linux/unix/bsd) for ACTUAL scenarios that the users of the systems expect them to perform. Not sure how important LAME encoding is for BSD people - I know I don't use it for that. Not sure how many Ubuntu users run DB servers or firewall routers.

        Can anyone point me to someplace like this? Someplace where I can see, for example, DB driven dynamic pages served per second - concurrent DB users, scalability to 4+ processors, etc?

        I want metrics like SAN performance, DB performance, mail server performance, HA, clustering, firewall performance etc for server "distros" (Windows Server, AIX, Solaris, BSD, Linux, OSX Server)
        I want metrics like graphics benchmarks, app response time, those classic GUI task tests for desktop "distros" (Windows, OSX, Linux, BSD).

        It annoys me that I see benchmarks that are so narrow and pointless. I can't make reasonable non-fanboy type comparisons when the benchmark doesn't include anything that matters to me.

        • Re: (Score:2, Insightful)

          by Anonymous Coward

          True. And let's also see some benchmarks with the system under heavy load. Run it as a webserver with thousands of page requests a second. Run it as a database server under a realistic load (not simply inserting a few records).

          I could give a rats ass about how fast it compiles some app, show me some REAL benchmark!

          • Thank you. That's my point exactly.
            Benchmark testing is very unscientific it would seem, since there is rarely ever a "control" and the test suites seem to run nearly random load sets.

            They're also almost always impossible to reproduce since the test suites are typically quite expensive... and many companies forbid publishing benchmarks!

            It'd be nice if a 3rd party like W3C did a webserver comparison where every company got to bring their big guns (each server configured by the *vendor* for best performance o

    • Re:What's the point. (Score:4, Interesting)

      by Jurily (900488) <jurily@noSpAM.gmail.com> on Monday September 28, 2009 @09:13AM (#29564601)

      I'm sort of curious what the point is of comparing an alpha to a release candidate. Additionally it's a minor update versus a major update. Throwing in an older release makes it all the more pointless as I'm not seeing anywhere in the summary that they disabled debugging.

      They left out almost all distros, too.

      • Re:What's the point. (Score:5, Interesting)

        by hedwards (940851) on Monday September 28, 2009 @09:20AM (#29564661)
        Well, with the number of distros out there, it's inevitable that they'd leave out most of them. But I must admit that leaving out every other distro makes it kind of hard to know if Ubuntu is doing better than the pack or about the same.

        It's kind of suspect, in my opinion, that the older release was doing so much better than the newer one, considering all the time that's been spent in recent times on optimizing various portions of the source. It's also worth noting that probably a much larger portion of the FreeBSD user base will recompile their kernel pretty much immediately with basic optimizations and removing the cruft that they don't need or want.
        • Re: (Score:3, Interesting)

          by fbjon (692006)
          Is there any actual benefit to be gained from removing "cruft", other than saving a smidgen of memory?
          • Re:What's the point. (Score:5, Informative)

            by geminidomino (614729) * on Monday September 28, 2009 @10:42AM (#29565615) Journal

            Is there any actual benefit to be gained from removing "cruft", other than saving a smidgen of memory?

            Long, but not long enough answer:

            Performance: Unless the cruft is a bunch of data or NOPs, it will be executed at some point, which is pointless (or it wouldn't be cruft.) And whether it's data or instruction, if "good" data is swapped out of the cache in favor of the cruft, then it will have to be read back in (cache misses).

            Security: Bugs love to hide in cruft.

            tl;dr version: Yes.

            • by compass46 (259596)

              I don't believe commenting out a bunch of device drivers from GENERIC for hardware that is not even in the box will have much of an overall effect really. Yes, there are a handful of other options in GENERIC but how many of those do ppl actually comment out and how much of an effect is there really?

              • by BitZtream (692029)

                Very little effect. The drivers are generally detached from the kernel if not used. Most things are loadable modules now anyway unless they are core components or so generic that EVERY PC has them.

          • by Abreu (173023)

            For want of a nail the shoe was lost.
            For want of a shoe the horse was lost.
            For want of a horse the rider was lost.
            For want of a rider the battle was lost.
            For want of a battle the kingdom was lost.
            And all for the want of a horseshoe nail.

    • by jhol13 (1087781)

      The comparision might have made sense if they had tested whether the different version of the compiler used is the "culprit" or not.

    • Re:What's the point. (Score:4, Interesting)

      by kad77 (805601) on Monday September 28, 2009 @11:21AM (#29566147)

      I'm sort of curious why the slashdot story summary is so annoyingly biased in it's phrasing - "FreeBSD rather ends up taking a wallop to Ubuntu Linux, but there are a few areas where FreeBSD 8 ran well", when the arguably flawed test suite shows NO SUCH THING!

      The FreeBSD system has very comparable or better benchmarks on nearly every metric in the test, just click through TFA and see for yourself.

      Tripe.

      Besides the needless and counterproductive bias, the phrase X "rather ends up taking a wallop to" Y is clunky and sophomoric. Editors, get a life.

  • by Anonymous Coward

    It's interesting how on page 1, there are three graphs. Two of these are "lower is better" (where Ubuntu wins), however, when FreeBSD wins the graph is displayed in MIPS where "higher is better", thus appearing to make Ubuntu win there too.

    If you're a casual reader not paying attention, reading, or clicking on to page 2 (and you know I'm right when I say that's most of the people reading this article), you can see where this is going.

  • Summary (Score:5, Insightful)

    by KingOfGod (884633) <ldrada@@@gmail...com> on Monday September 28, 2009 @09:06AM (#29564547) Homepage Journal

    The article is of little use other than tell the general populace of Slashdot that FreeBSD 8.0 and Ubuntu 9.10 are right around the corner, and that we should be hyped. Also FreeBSD 8 is a little faster than FreeBSD 7.2 but a lot slower than Ubuntu Linux 9.10

    I'm not surprised, however I do belong to the group that does not really care about relative performance to other OS's as performance is only one of the aspects from the vector of decisions we had to make to finally choose FreeBSD for mass-scale deployment.

    • Re: (Score:3, Insightful)

      by mi (197448)

      Also FreeBSD 8 is a little faster than FreeBSD 7.2 but a lot slower than Ubuntu Linux 9.10

      Not disputing the conclusions of the memory, I/O benchmakrs, part of what they benchmarked is the compiler — FreeBSD's gcc-4.2.1 vs. Ubuntu's 4.4.1. I'm not surprised, GCC's newer release both compiles faster and produces faster binaries.

      You could say, FreeBSD is at fault (and thus deserves bad rep) for including an outdated compiler, but, on the other hand, FreeBSD's choice may prove wiser, when bugs in the ho

      • Re:Summary (Score:4, Informative)

        by TheRaven64 (641858) on Monday September 28, 2009 @11:35AM (#29566375) Journal

        FreeBSD ships with GCC 4.2.1 as the system compiler because it was the last release to be GPLv2 and GPLv3 stuff is not allowed in the base system. GCC 4.4 is in ports, and you can use this to compile ports easily by just setting a flag in make.conf if you care. FreeBSD 9 will hopefully be using LLVM/Clang as the system compiler, which should give it a nice boost.

        Phoronix has a history of doing long and misleading benchmarks between Linux and *BSD/Solaris, where they manage to include so many extraneous factors that the results are meaningless.

  • Benchmarks... (Score:3, Insightful)

    by coryking (104614) * on Monday September 28, 2009 @09:07AM (#29564559) Homepage Journal

    Benchmarks are useless. There are way more important things to judge an operating system then "speed".

    Does Ubuntu have nearly as good of documentation? No. It has that "info" nonsense.

    Does Ubuntu provide a stable platform to build a server? No. It, like most linux distros, changes whole versions during updates. That isn't stable.

    Does Ubuntu provide a way to strip itself down to the bare metal? Ain't as easy as the BSD's.

    Is Ubuntu built around solid engineering and design, or politics? Depends--Ubuntu seems to be less afraid of the big bad FSF as other distros, but it still is steeped in an OS built for politics. FreeBSD is pretty tame and tends to focus on solid engineering rather than political maneuvering.

    But really, Comparing FreeBSD to Ubuntu is like comparing OpenSolaris to Windows 7. FreeBSD is largely a server operating system were as Ubuntu is an end user operating system. And if you are comparing server operating systems, there are far more important criteria than "speed". Things like version stability are vastly more important.

    • Re:Benchmarks... (Score:5, Informative)

      by nxtw (866177) on Monday September 28, 2009 @09:31AM (#29564783)

      Does Ubuntu provide a stable platform to build a server? No. It, like most linux distros, changes whole versions during updates. That isn't stable.

      Ubuntu has LTS (long-term support) releases which are supported for 5 years on the server side. The last was 8.04 and the next will be 10.04.

      I prefer RHEL/CentOS, however. I wonder how many people use Ubuntu LTS instead of using RHEL or SLES instead.

      Does Ubuntu provide a way to strip itself down to the bare metal? Ain't as easy as the BSD's.

      How often is this important? I can think only of a few situations, such as when fitting a system into a small/cheap flash.

      But really, Comparing FreeBSD to Ubuntu is like comparing OpenSolaris to Windows 7. FreeBSD is largely a server operating system were as Ubuntu is an end user operating system. And if you are comparing server operating systems, there are far more important criteria than "speed". Things like version stability are vastly more important.

      Ubuntu has a separate 'server' version (which really just includes a different set of packages and a different kernel build.)

      • Ubuntu has a separate 'server' version (which really just includes a different set of packages and a different kernel build.)

        Wait.. what else is there?

      • Re: (Score:3, Insightful)

        by BitZtream (692029)

        How often is this important? I can think only of a few situations, such as when fitting a system into a small/cheap flash.

        Its important to those of us that know by not including extra stuff on the machine, that extra stuff can not possibly be exploited, cause problems, waste resources or any number of other reasons that don't come to mind right at the moment. Including extra stuff is almost always a bad idea when you're doing a server. Desktops are a little different. Interestingly enough, Ubuntu is desk

    • Re: (Score:3, Insightful)

      1. Agreed, I think neither is really better and use depends on what you need, which of course some Linux and BSD zealots seem to disagree on - apparently the main free *nix are now entrenched enough to be part of the unix holy wars.

      2. Varies, FreeBSD doesn't have perfect docs either and other distros do have better docs

      3. Lrn2LTR

      4. Yes, it does, there's a 10MB barebones installer for the dedicated and if you need less, pick another distro, even FreeBSD will probably be tons of tweaking at this point.

      5. Poli

    • Re: (Score:3, Insightful)

      by moon3 (1530265)
      Speed is the most important factor for web-server scenarios, if FreeBSD can handle 10x more inserts into SQL-Lite then Ubuntu in the same benchmark, on the same hardware then Ubuntu is KO'ed by BSD in the server arena, no offense.
      • by AusIV (950840)
        I am curious how much difference there would have been on a database system that would actually be running in a server environment. Ubuntu definitely got owned by that metric, but SQLite is hardly exemplary of a production database.
    • Re: (Score:3, Insightful)

      by Rogerborg (306625)

      Ubuntu [is] steeped in an OS built for politics

      Hah, I see what you did there. You ripped on Ubuntu for being steeped in politics, while ignoring the available facts, i.e. that Ubuntu 10.4 is a stable Long Term Support release, and that it wipes the floor with FreeBSD 8.0 in performance.

      That's pretty funny. I mean, if you did it on purpose.

    • by dkleinsc (563838)

      the big bad FSF

      I must be missing the part where the FSF became the evil empire who wanted to control the world of computing, rather than a rebel alliance who wanted the users in control of the code running on their own computers. And the part where the FSF did anything to actively stop FreeBSD from doing its thing.

      The Ubuntu project, the Linux project, and the FreeBSD project all have the same basic goal: create freely shareable really awesome (fast, effective, efficient, robust, useful, and as bug-free as possible) code

      • by Abreu (173023)

        I must be missing the part where the FSF became the evil empire who wanted to control the world of computing, rather than a rebel alliance who wanted the users in control of the code running on their own computers.

        The change probably happened when Obama became president and suddenly transformed from a "beacon of hope and change" to "a scary communist who wants to take away your freedom"

    • by nimbius (983462)
      Hey! benchmarks are too useful!

      without these numerically-backed conslusory points, id be forced to attempt to justify budgeting and purchases to the PHB using the potential usefulness and benefit of a service to its userbase. giving an old man a line-graph and pie chart with made up numbers means he doesnt need to know anything more than red is bad, green is good.
    • You sounded like you had a good point, until you let fanboy rants cloud it.

      Hey, politics ain't just for the mainstream, y'know.

  • by elrous0 (869638) * on Monday September 28, 2009 @09:08AM (#29564561)
    When my friends ask me about Linux, I usually steer them toward Ubuntu first, as it's the most user friendly and well-supported distro out there. Canonical really puts a lot of development work into it, and it shows (in this result and many others). In the past, I usually avoided the Linux topic altogether, as there were so many confusing distros that even trying to explain the concept of Linux to non-geeks (and even many geeks) was a huge pain in the ass. So, I for one welcome our new Ubuntu overlords.
    • I recently wanted linux on a HP laptop. I tried the latest Ubuntu disk and it just didn't load. I decided to try Mandriva and things worked great. It was the first linux install that I didn't have to do any tweaking for video or wireless. I was pleasantly surprised.
  • by MikeRT (947531) on Monday September 28, 2009 @09:13AM (#29564599) Homepage

    The areas where FreeBSD gets its ass kicked by Ubuntu start on page 7...

    It seems to me like FreeBSD's real problem is incredibly bad I/O compared to Linux. The majority of the CPU-heavy tests were nearly neck-in-neck.

    • Re: (Score:3, Informative)

      by amorsen (7485)

      It's fairly difficult to bungle traditional CPU-heavy loads. The kernel just needs to get out of the way and let userspace do whatever it wants to do. Try the same tests on Mac OS X and Windows (assuming they compile), and you'll see just about the same performance.

    • by Galactic Dominator (944134) on Monday September 28, 2009 @09:35AM (#29564809)

      That's to be expected considering the defaults of ext4 vs ufs2. You can increase flush time on ufs2 and expect a similar increase. Revert to ext3 and it would be a completely different outcome. Interesting to see all the chest pounding on choice for default settings in a desktop enviro vs a traditionally server one. Would have been a been comparsion to use the upcoming PCBSD's release vs Ubuntu's, but we've seen the bias from Phoronix before.

      • by Bralkein (685733)
        FTFA:

        With Ubuntu 9.10 we were using the x86_64 server CD of the Alpha 6 build. With FreeBSD not shipping with a desktop environment by default, we used the Ubuntu server CD so that both could be tested just from the terminal in a similar environment.

        So they are comparing FreeBSD to the Ubuntu server version, but not really for the right reasons.

        • I suppose that's a bit closer, but it still doesn't address the differences between fs defaults. ext4 is far more aggressive than ufs2 is by default. A closer comparison would have be to gjournal ufs and mount async which would have been a relatively close comparison to ext4, but that still would not be a fair io comparison as gjournal scales incredibly well but can double write time on single write. I am struggling to comprehend ext4 as a default filesystem in a server(or anywhere) however, considering

          • by Bralkein (685733)
            Yeah there are loads of configuration options that are relevant on servers. Going with the defaults might be sensible when comparing desktop OSs (since Mr Noobey doesn't know how to select a more suitable IO scheduler) but we've already established that they're looking at server OSs here. I think Phoronix show here that they really haven't thought this comparison through. Once the competence of a benchmarker is called into question all of a sudden I feel a massive urge to completely ignore everything they s
            • by Kjella (173770)

              Once the competence of a benchmarker is called into question all of a sudden I feel a massive urge to completely ignore everything they say.

              You mean like pretty much every benchmark where there's a sore loser? Well, I'm sure they'll be glad to hearing spreading FUD is such an effective strategy. I'm not talking about this specific case, but for benchmarking in general your position is rather sad.

              • by Bralkein (685733)
                Perhaps "called into question" wasn't the best choice of phrase. I certainly don't mean that if absolutely anyone, anywhere cries BS about a benchmark then the benchmark is instantly void, rather I mean that once a benchmarker starts to look like they don't know what they're doing then since I don't have enough time to pick through every inch of their technique and raw data, it's easier just to file the whole thing under "useless" in my brain and move on with my life.

                The problem is that there are so many
      • It's worth noting that one of the things in RC1 is that ZFS is now considered 'production ready' by the FreeBSD ZFS team. Maybe they should enable it for FreeBSD and then add a benchmark of shapshot and transactional I/O performance...
  • by vrmlguy (120854) <{samwyse} {at} {gmail.com}> on Monday September 28, 2009 @09:19AM (#29564651) Homepage Journal

    Eight pages of bar charts, each gray-on-gray. On half of them, shorter bars mean better performance, on the other half, longer is better; the only way to know which is which is in a legend, written in a small font.

    Here's a suggestion: color-code the bars! Green is good, red is bad, yellow is in the middle of the road. For bonus points, choose the saturation based on magnitude of the differences. If the numbers are close, go with grayer bars, if the differences are dramatic, use dramatic colors.

    Finally, how about a line chart at the end showing all of the numbers in one place? Yeah, you'd need to convert everything to be consistent if longer or shorter is better, but that's a good idea anyway.

  • BSD did rather well (Score:4, Interesting)

    by John Jamieson (890438) on Monday September 28, 2009 @09:21AM (#29564677)

    I'm not a BSD user, but I don't see BSD taking a real kicking in these benchmarks. In the majority of the benchmarks, the average user could not discern a speed difference.

    • by canajin56 (660655)
      Look, this is benchmarking. You need to know your lingo. Take 3D benchmarking. Even though most monitors will have your FPS capped at 60 Hz, if one card gives you 250 FPS and the other only gave you 240 FPS, you say that the new card "absolutely destroys the competition," for example. Anything even partially outside the error bars translates into at least a "solid thrashing."
  • I'd actually like to see a chart with these types of performances going back a number of years. It would be interesting to see the difference between RH10 and the Latest Fedora release (assuming you could even get RH 10 to run on modern hardware without backporting drivers). Same wit FreeBSD.

  • by Just Some Guy (3352) <kirk+slashdot@strauser.com> on Monday September 28, 2009 @09:39AM (#29564831) Homepage Journal

    From the update notes in /usr/src/UPDATING:

    NOTE TO PEOPLE WHO THINK THAT FreeBSD 8.x IS SLOW:
    FreeBSD 8.x has many debugging features turned on, in both the kernel and userland. These features attempt to detect incorrect use of system primitives, and encourage loud failure through extra sanity checking and fail stop semantics. They also substantially impact system performance. If you want to do performance measurement, benchmarking, and optimization, you'll want to turn them off. This includes various WITNESS- related kernel options, INVARIANTS, malloc debugging flags in userland, and various verbose features in the kernel. Many developers choose to disable these features on build machines to maximize performance. (To disable malloc debugging, run ln -s aj /etc/malloc.conf.)

    Since the article didn't mention anything about disabling all the debugging options, I'll consider this an invalid benchmark until shown otherwise.

    • Re: (Score:2, Informative)

      by Anonymous Coward

      WITNESS and INVARIANTS were disabled on Sep 10 (http://svn.freebsd.org/viewvc/base?view=revision&revision=197065). FreebSD 8.0-RC1 was built on Sep 17th.

    • by Cato (8296)

      Mod parent up even higher... in fact mod down the ones of equal score, this is probably the only comment that matters. It's unlikely Phoronix disabled all the debug features (would probably need a system recompile) so the benchmarks are most likely a waste of time.

  • I which I'd seen this before I posted my last comment:

    FreeBSD 8.0 vs. Ubuntu 9.10 BenchmarksFreeBSD performed awful in comparison to Ubuntu when doing random writes, where the latencies were extremely high and off the charts compared to Mark Shuttleworth's operating system.

    How "awful" for FreeBSD and good for the OS that Mark wrote. My grammar and clarity aren't always perfect, but hey, I don't get paid to write this stuff. :-)

    • but hey, I don't get paid to write this stuff. :-)

      Oh, admit it -- sure you are, it's just that it's by an employer who doesn't realize that's what he's paying you to do. ;-)

  • by LizardKing (5245) on Monday September 28, 2009 @09:49AM (#29564961)

    Yet again a benchmark against a pre-release version of FreeBSD where the testers didn't even bother reading the documentation. Anyone actually familiar with the FreeBSD development and release process would know that a release candidate has a considerable amount of debugging options turned on. This is to help diagnose any problems as the last issues are shaken out of a release, but has an adverse impact on performance.

    • by kill-1 (36256)

      Yeah, everytime I read one of their articles "Moronix" comes to my mind. But as long as they are the only ones who regularly benchmark Linux (and sometimes BSD) systems they have a kind of monopoly.

    • Re: (Score:3, Informative)

      by Conley Index (957833)

      Anyone actually familiar with the FreeBSD development and release process would know that a release candidate has a considerable amount of debugging options turned on.

      On Sep-10, most debugging was disabled: http://lists.freebsd.org/pipermail/svn-src-all/2009-September/013399.html [freebsd.org]

      On Sep-17, there was the last commit before 8.0-RC1: http://lists.freebsd.org/pipermail/svn-src-all/2009-September/013645.html [freebsd.org]

      Anyone familiar with the FreeBSD development and release process would know that there are no fixed rules rules when certain stuff happens and there are no sweeping changes like turning off debugging between a late RC and the actual release. (Other debugging stuff like ker

  • by buddyglass (925859) on Monday September 28, 2009 @09:53AM (#29564999)
    I stopped reading when I realized they didn't even use the same version of GCC in their compilation comparison.
  • Interesting comparison but the SQL test was a bit off testing a database installed on ext4, a journalling filesystem. Anyone wanting intensive database work would use ext3 for the DB partition.

    Also, it would be interesting to see whether FreBSD still outperforms Linux in low-memory situations. This used to be an area where BSD had a clear advantage.
  • The conclusion of the benchmark seem pretty unclear to me. comparing a development versin with debugging compiled with different compilers on different operating systems lead to different result.

    I could have guessed that. I believe a benchmark is useless without a good insight of why the performance are different and how to correct the problem.

  • FreeBSD rather ends up taking a wallop to Ubuntu Linux, but there are a few areas where FreeBSD 8 ran well.

    Apparently taking a wallop to != bringing the hurt.

    Perhaps it would be clearer to say For the most part, Ubuntu beat FreeBSD, or perhaps FreeBSD got walloped by Ubuntu.

  • Some of the code in BSD family of code streams is so old its growing fungus. Still, when I set up a server I usually reach for OpenBSD, its proven rock-stable to me after years of trouble-free mail, firewall/NAT, and web server service. But Ubuntu's my desktop.

    • I do much the same as you. The BSDs make excellent servers, firewalls, and routers. Although, I do use PC-BSD for my desktop. Comes down to personal preference. I use OpenBSD for my firewall, router, and vpn appliance. FreeBSD serves up my web pages, ftp, dns, and just about everything else.
  • While, interesting the test wasn't really comparing apples to apples. For example, FreeBSD RC 8 is using a slightly older version of GCC so the first test is somehwat invalidated. It is also notwworthy that several of the tests are canted towards desktop use. FreeBSD, you could argue, is not as optimized towards desktop use as Ubuntu. A better test would be to compare the Ubuntu Server edition with FreeBSD. I would like to see a test designed around the number of simultaneous website hits either OS can
  • by cracauer (6353) on Monday September 28, 2009 @01:18PM (#29568103) Homepage

    People mentioned the self-checks and debugging features that used to be turned on in FreeBSD development branches and beta releases.

    Self-checks, which are the major source of kernel slowdowns in those kernel options, are not turned on in the 8.0 release candidates.

    Debugging is on, but unless you are very short of memory it should not cause a noticeable slowdown.

    FreeBSD's slowness in these benchmarks can be attributed to two factors:

    1) the compiler. The GPL v3 is unacceptable for FreeBSD, so newest GCCs cannot be used as the base compiler. Users can choose to install a new gcc on their own (as a port) and then even go and compile all ports and even the base system with that new compiler (some parts might not have been cleaned up to comply with new language strictness that might have come with new gccs).

    2) threading, as in the userland threading library support. It is very hard to tell whether there is some performance problem in FreeBSD's thread libraries or whether applications just happen to have been optimized and tested only with Linux.

    That happens a lot and you can also see Solaris with it's M:N threading fail miserably for some threading benchmarks that do dump things, such as just creating and destroying threads at a high rate.

    %%

    The problem of "thread benchmarks" benchmarking bogus things and/or just having been written with Linux' thread model in mind has been going on for 12 something years now. Benchmarking thread systems in a realistic manner is very difficult. In real-world applications you don't create and destroy threads at a high rate and you minimize locking. Benchmarking this is almost as hard as benchmarking programming languages.

    I haven't benchmarked threading libraries, knowing that it will take time that I don't have right now. I can tell you, however, that just the I/O subsystem in FreeBSD, as in filesystems and networking, isn't any slower than Linux. Not to mention GbE and today's disks are too slow to really show an OS difference for most tests anyway.

    %%

    The real question of I/O subsystems will come in when ZFS+Zraid is standard in FreeBSD and BTRFS is standard in Linux. In a couple of years from now nobody will understand why we ran today with no snapshots, with the raid hole (from block layer raid systems) and without transparent compression in the subtrees where we want it.

    But these filesystems are complicated and there's some real performance difference visible.

    %%

    An area completely overlooked in the benchmark is the VM subsystem. Namely - what happens when you overload your RAM and paging commences? Linux used to make very bad choices here (dropping readonly pages, which is a wise thing as such, at rates about 10 times higher than wise).

    FreeBSD used to be the go-to OS on memory shortage thanks to John Dyson's VM work (backed by a large database company that provided support and a realistic benchmarking environment during that development).

    But today? Nobody knows. I'm not aware of any benchmarks that you can download that simulate memory stress and map the tradeoffs that the OS makes.

    In general, the biggest obstacle to improve Linux, FreeBSD and everybody's else's OS performance is a lack of high-quality benchmarks.

    Why don't people develop more benchmarks? Because they get annoyed that today, in 2009, no realistic OS benchmark will show a single number as a result. All OS benchmarks today can only make a map of what tradeoffs the OS chose, what part of the running application suite got the short end in favor of what other part. This isn't sexy and publishers don't like it.

    People like reality reduced to single numbers, but in the area os OS benchmarks (and language benchmarks) that party is over.

    Myself, I found myself gasping many years ago when I benchmarked network I/O versus userland CPU load. I hammered a couple of GbE interfaces while at the same time running moderate memory-intensive CPU benchmarks (with no network access from those CPU lo

"A mind is a terrible thing to have leaking out your ears." -- The League of Sadistic Telepaths

Working...