Forgot your password?
typodupeerror
Linux Software

The Mindcraft Debacle: Part MCXVI 137

Posted by Hemos
from the more-fun-with-numbers dept.
Quite a number of people wrote in to alert us to the Salon story about the Mindcraft survey. Andrew Leonard, does a great job of tracing the trail, including the information about how Mindcraft is asking for help from Linus, but is not apparently giving him the information that he needs. Ah, the joys of industry. Mindcraft has agred to re-do the tests, with Linus' help, as aforementioned, but is continuing to be...less then forthcoming about the tests. Meanwhile, Microsoft is trumpeting the original study.
This discussion has been archived. No new comments can be posted.

The Mindcraft Debacle: Part MCXVI

Comments Filter:
  • by Anonymous Coward
    Come on, guys. The writing is on the wall. Microsoft has been pumping up Linux in all sorts of subtle ways. They don't want to be caught actively pumping the software, but DAMN, they're doing just about everything they can otherwise. Drawing media attention? Leaked internal memos? Porting software? Horribly biased tests?

    Microsoft has never made such serious mistakes in so short of a period of time with the rest of its competitors.
  • It would be humorous if microsoft got slashdotted by all of us trying to read theyre anti linux propoganda saying that nt is a better web server.
  • It's like Alan Cox said in the Salon article, they hope to be able to say something like, "NT beats Linux by 600x with aid from Linus Torvalds himself!!" conveniently not mentioning that it was limited access, not full information, and Linus only able to give guesses and suggestions.
  • by drwiii (434)
    "Microsoft Daily News" [microsoft.com]? And I thought those weirdos over at The 700 Club [cbn.org] were biased..
  • C2 certification is not mandatory for all government systems, only for few of them. However POSIX is required, and this is why Windows NT has POSIX subsystem, even though no one uses it because of its extreme brokenness.
  • by Steve Philp (507) on Tuesday April 27, 1999 @10:37AM (#1913600)
    Can anyone else picture Gates wringing his hands at the pure joy in seeing the results from this round of testing?

    "Even with help from the top two Linux minds, namely kernel inventor Linus Torvalds and top lieutenant Alan Cox, Linux was unable to match the performance of the NT machine under test."

    If Mindcraft is being as unhelpful in giving information to these two as they were in replying to Usenet posts for help, you're going to see a steaming pile of FUD right in the middle of that report.

    Let's face it, the Mindcraft results are going to say EXACTLY what the Microsoft marketing forces want them to say. If the marketing people say there's a perception that Linux is difficult to get support for, you can damn well bet you're going to see forced substantiation of that claim in their report notes.

    This second test, if it continues under the same sort of conditions that Salon notes, will NOT be any better than the previous. In fact, given the level of expertise being "consulted" in the new test, this result could hurt the Linux image much, much more.
  • The original test used quad-Xeon machines and I'm
    sorry but I think that's unrealistic -- how many
    shops use quad-Xeon-processor machines for web serving
    and file serving? I think dual-PII-processor is the
    more common workhorse in this industry, and I'm
    pretty confident Linux beats NT on dual-processor
    hardware.

    They should be testing "typical" hardware
    platforms rather than using hardware so maxed-out
    that hardly anyone would own such a machine
    these days, or at least not for plain-ol'
    web serving -- database serving maybe but not
    for web serving and file serving -- that's
    ridiculous!

    ...and I think the whole point of Linux is you
    can run a pretty fast web server on *cheap*
    hardware so you really don't need quad-Xeon
    w/ 1 GB of RAM to serve some serious enterprise
    level web traffic anyway -- a plain ol' pentium box
    will work just fine.
  • This sounds like another MS setup to make Linux look bad, and attach Linus and Alan to the "failure" of Linux.

    What we need is an open test conducted by a trusted third party. MS can show up with their 50 drones and go head to head against a handful of Linux/Samba/Apache developers.

    That would be something to read about, instead of all these bogus benchmarks payed for by Bill.

    TedC

  • It's really refreshing to see the power of the 'net in rooting out the obviously slanted "truth" that MS would have folks believe. Again, this whole debacle just proves that MS still doesn't "get it" about the Internet/WWW/usenet. When you put something out for mass peer review, it's going to get hammered. Well, MS just got the bejeesus pounded out of it.
  • Quoth Ed Muth, group project manager at MS:

    "Microsoft was pleased but not surprised by Mindcraft's results concerning the excellent performance of Windows NT Server."

    Why am I not surprised either.
  • RAID 4 is the dog-ass-slow write performance because there is only one parity disk. RAID 5 is striped parity with the pairity bits hitting each disk. It is slightly slower than a single drive for writing, but significantly faster than mirrored drives (the slowest write of all RAID systems).

    RAID 5 is the 2nd fastest reading system (losing only RAID 1 across an equal number of disks and, for example, 5 disk RAID 5 is basically equal reading to 4 disk RAID 1).

    If you have hardware RAID like these servers, then you're not going to notice anything with write speed.

  • Each disk you throw in slows your writing down a lot. Not so with RAID 5.
  • They had a god-awefully small StartServers and MaxServers line as well as a MaxSpareServers 1. They also ran apache out of inetd.

    They did *NOT* tune apache upward.

  • RAID 0 is striping. In fact, RAID 0 will speed up writes and reads if it is implemented correctly. Many companies don't do this and simply create a volume set so the 2nd and later drives don't get written to until the first is full.

    RAID 1 is mirroring. That means everything you write to drive 1 gets written to drive 2 and drive 3 and drive 4, etc. That's X copies getting written where X is equal to the number of drives.

    RAID 5 is striping with striped parity. Unlike RAID 4 which places a bottleneck on the parity drive, RAID 5 spreads the parity across all drives. Parity calculation is often done very fast in hardware, like on those *hardware* RAID controllers.

    Now you've just gotten almost an hour worth of CS lecture in 3 paragraphs. If I have to do that again, I'm going to bill you.

  • It's very obvious you don't know how RAID 5 works.

    First, yes it's an XOR (at least you got that right). However, reading takes just over x+1/x amount the time of reading from a RAID 1 set. Just over being the time it takes to do an XOR and comparison.

    Writing, however, takes x+1/x amount the time as it takes to write to a true RAID 0 (aka striped, no parity) set. Again, the extra time is the time it takes to calculate the parity (XORing is very cheap, in fact, on the i386 set, xor ax,ax is faster than move ax,0).

    So, again, what was that?

  • How do you test against your parity information?

    You lose.

  • ...did nothing wrong, we should believe them?

    If I am wrong, it should be trivial to correct me. No one has.

  • by Matts (1628)
    I don't know where you got your ideas from.

    If you're looking for optimal performance you have to a) know what you're doing. and b) compile the packages yourself, including only the options you need.

    I also don't know where you got your ideas about RAID. RAID 1 is what you want for higher performance on a web server. RAID 5 shows little thought went into the choice.

    perl -e 'print scalar reverse q(\)-: ,hacker Perl another Just)'
  • by Matts (1628)
    I'd run RAID1 on a real server - in a second.

    Perhaps you need to read up some on RAID - RAID1 is the fastest and safest of all the RAID standards. It's what you use if cost is not an issue. For a server, that tends to be the case.

    For some good introductory docs on RAID try the Software-RAID mini HOWTO.

    Matt.


    perl -e 'print scalar reverse q(\)-: ,hacker Perl another Just)'
  • by Matts (1628)
    I think they were aiming with telescopic sights... We managed to get twice the performance they did out of a SINGLE PIII500...

    perl -e 'print scalar reverse q(\)-: ,hacker Perl another Just)'
  • I see mindcraft trying to pull itself out of the ditch it threw itself into by strutting it's stuff about NT beating the pants off of Linux, but I have a comment:

    They documented that they did some serious tweaking of the NT server, even so beyond what microsoft suggests for reliabilty. They had to get that information from somewhere, they probally had a MS tech either over their shoulder or on the phone telling them to apply this reg tweak, apply this patch to bind the nic's to each CPU....

    Why can't we do the same for linux? Why can't Linus and have direct access to the server that they're supposed to be helping on?

    They must be trying very hard to "proove" that NT is "better" then Linux.

  • " shoudl work

  • Instead of trying to beat Microsoft and Mind-benchmark-craft in a stacked environment, we should build and tune a Linux box that provides equal or greater performance with less expensive hardware than the Dell.

    Mindcraft did publish the performance of their NT box, so there is a target to aim for.

  • The reason Mindcraft is calling on Linux and Alan for help is to recover their own lost legitimacy. The mainstream tech media is openly calling the integrity of their work into question. If, as some worry, they try to attach Linus and Alan's names to another rigged test without giving them proper access to the test environment, then they risk being denounced by Linus and Alan themselves... which would almost certainly be reported in ANY press coverage. That would just expose the tests for what they really are - paid FUD.
  • The Customers, the gray mass that believes everything MS says, will be fooled, as are those who don't read any news from other sources. MS will fool a few people with that trick, especially, well, MS-drones.

  • You missed this interview with Linus some time ago where he said (sort of late answer to Tanenbaum's "Linux is obsolete"), mainly, that microkernels were the panacea for researchers only, but that in real world you could have most of microkernel's benefits without the overhead with a monolithic kernel and good code design...
  • Before this Linux testing disaster, Mindcraft had performed similar comparisons between NT4/NetWare 5, NT4/NetWare4.11 and NT4/Solaris. They are, historically, a professional, unbiased testing shop who can accurately perform tests that will produce whatever Microsoft wants - Microsoft seems to be their largest paying benchmark customer. They also did some tests for Netscape, too, "proving" that Netscape blows away NetWare for directory management.

    You'd think that if a company pays for tainted benchmarks, they could at least ask Mindcraft to cook the numbers less blatantly.

    THe best way to do this test, I think, would be to run the NT benchmarks, then let Linus, Alan, Brian, Andrew, and Dean (I apologize for any gurus I forgot) have their way with the same server for testing. Let the Mindcraft stooges watch. They may just learn something.
  • by datazone (5048)
    Since RedHat and and other Linux based companies have the most to lose from this, i suggest that they do a test on similar hardware and publish everything that they did, and what results they got.

    However waiting for a company to do something is a stupid way to go. I suggest some non-profit linux organization find out how much it will cost to purchase one of these machines, and create a fund where people can contribute money towards its purchase. Then allow folks who know about configuring it contribute information on how to do it. Publish the report, and bam, end of story. The machine could then be used for other linux related things such as hardware for linux developers to work and optimize on, or other such things. The potential for this is great since if its alot of folks will be willing to contribute to it. hell i got $10 bucks to go towards this beast.
    and i am sure that the money could be raised from slashdot users alone, much less the entire community, since once the project is started, companies will try to get some press coverage by saying they helped in the project.

    Anywho this is my humble opinion, and what do i know?

  • I am sure that we will do better, I wonder if they changed the hardware? I am not sure that
    with the AMI RAID driver will do much better, but everything else should work.
    I don't expect them to change hardware, and to be honest I don't want them to, it's up to us to
    be real world too.

    Three things I hope they do:

    Use kernel 2.2.pre7 patch3, it's a very nice kernel.

    More RAM, at least to the two gigabyte limit.

    One OS on the drive, and real world raid5, not raid0, this is a server stupid.

    The right amount of swap I have wondered and wondered about that one.

    A correctly tuned apache server, at least use the .rpm, do not compile unless you really
    know what your doing.

    SAMBA, use the damn package unless there is a GOOD reason not too.

    Get rid of the enterprise edition; server is what most people will buy it's a bit more real world.

    And last document better; I would feel better if I new exactly why something failed or not.


  • by law (5166)
    If it was ME that was doing the testing; I WOULD compile, compile, compile.
    But it's Mindcraft, I would not want them too.
    It's a SERVER!!
    Not just a 'Web server' both should be running RAID 5.
    Think about it; would you run RAID1 on a real server?
    If you do; I don't think I would trust you with my data.
  • Very Right! Could get even worse!

    My take is that anyone from the Linux community participating in the "fairer" (what a joke) test should instead release a disclaimer stating the fact that any test that does not include colaboration from Linux people is invalid!!

    I suggest Linus, Alan Cox or anyone from Linux simply put out an open letter stating their discontent with the issue of secret tests, and avoiding being pawns in M$ propaganda machine.

    Branman
  • It worries me to read in the Salon article than Linus and Alan Cox are trying to give advice to Mindcraft for a second test, but Mindcraft isn't giving them any real access.

    It worries me because I think the Linux community did a very good job of refuting the ridiculous results of the test, but if they redo them and can claim that Linux lost even with the help of Linus and Alan Cox . . .

    Allowing Mindcraft to do another test gives them a legitimacy that they don't deserve. I would much rather see VA research do a test, they would do an excellent job of tuning Linux, and in fact I would even trust them to be fair to Microsoft.

    I realize its not up to us to say whether they repeat the test, but it should just be ignored.

    Summary: we already debunked the Mindcraft benchmark, lets not play back into their hands by taking them seriously and giving them a renewed credibility. Please, VA research or someone of that ilk, come out with your own more legitimate tests.
  • by John Kacur (5703) on Tuesday April 27, 1999 @10:30AM (#1913628)
    It worries me to read in the Salon article than Linus and Alan Cox are trying to give advice to Mindcraft for a second test, but Mindcraft isn't giving them any real access.

    It worries me because I think the Linux community did a very good job of refuting the ridiculous results of the test, but if they redo them and can claim that Linux lost even with the help of Linus and Alan Cox . . .

    Allowing Mindcraft to do another test gives them a legitimacy that they don't deserve. I would much rather see VA research do a test, they would do an excellent job of tuning Linux, and in fact I would trust even trust them to be fair to Microsoft.

    Summary: we debunked the Mindcraft benchmark, lets not play back into their hands by taking them seriously and giving them a renewed credibility. Please, VA research or someone of that ilk, come out with your own more legitimate tests.
  • We should start a "why no widelinks" campaign. Most of the MindCraft misconfigurations could be written off as mistakes. The "widelinks = no" setting, contrary to the better performing default, smacks of actual sabotage.

    The only story getting out into the media is that Linux was not optimised. What's really going on is that in some areas it was pessimised.

    Spread the word!

  • I think you've both got your RAID numbers confused. There's certainly nothing wrong with using RAID 1 on a server.

    RAID 0 = concatenation/striping
    RAID 1 = mirroring
    RAID 5 = distributed parity

    RAID 0 by itself provides no redundancy at all, it just combines several physical disks into one larger virtual disk. It can improve performance, especially if striped properly. In fact, RAID 0 on its own reduces redundancy - if one disk goes, you lose the entire volume.

    RAID 1 is mirroring, which gives you redundancy, at the cost of extra storage requirements - you need two (or more) 1GB disks to get a 1GB mirrored volume. Mirroring can slightly improve read performance, and slightly disimprove write performance (everything has to be written twice, or more).

    A common setup is RAID 0+1, striping and mirroring in combination.

    RAID 5 is distributed parity - another form of redundancy. You can combine, say, 4 disks into one volume, with 3 disks worth of data and 1 disk worth of parity information.

    RAID 5 is cheaper than 0+1, since it requires less storage. Read performance is fairly good, but here's the catch: write performance is lousy. If it's a write-intensive filesystem, don't use RAID 5.

  • Yes, you're right when you say RAID 5 is striped parity - that's what I said too. But you're wrong if you think you're going to get reasonable write performance from RAID 5.

    Think about it for a minute.

    Every time you write a block on a RAID 5 volume, you have to recalculate the parity for that block. That means you have to reread the parity stripe, and XOR the information before you can write the block.

    Hardware RAID helps with the XOR process, but it doesn't help with rereading the disk.

    If you have hardware RAID like these servers, then you're not going to notice anything with write speed.

    If you're doing a lot of writes, you very definitely will notice decreased performance compared with mirroring.

  • It's very obvious you don't know how RAID 5 works.

    Oh, save the condescending attitude. I work with RAID systems every day. Don't tell me I don't know my job.

    You conveniently ignored what I said, so try to pay attention this time: to write a RAID 5 partition you must reread parity information from the disk, unless you are doing a Full-Stripe-Write; you need to write (ndisks -1 ) * stripe_size bytes of data in one operation to avoid the performance hit. Oh, and that data has to aligned on a stripe boundary, too.

    Otherwise, you end up doing either a Read-Modify-Write or a Reconstruct-Write, and that's expensive.

    For write-intensive filesystems, don't use RAID-5

    My source for this?

    Now are you going to tell Veritas that they don't understand RAID either?

    First, yes it's an XOR (at least you got that right). However, reading takes just over x+1/x amount the time of reading from a RAID 1 set. Just over being the time it takes to do an XOR and comparison.

    You're wrong there, too, by the way. Reading a RAID-5 volume is just as quick as reading RAID-0. No XOR'ing needs to be done when you read. And you have the cheek to accuse me of not understanding RAID-5?

  • I'd stop now, bkosse, if I were you. Have you noticed how every one is disagreeing with you?

    Could be because everyone in this thread except you knows what they're talking about?
  • Since Microsoft is using a knowingly flawed report (they admitted that the report was flawed in some South African periodical), there are good grounds to hold them responsible for false and misleading advertising. On that basis, everyone who reads this should file a complaint with the FTC at http://www.ftc.gov/ .

    Now if we could only get MS-lackey Jerry Pournelle to retire, we might actually get some quality journalism.
  • Comparing comparable Linux and Windows NT Server-based systems...

    How redundant. How does one compare two incomparable systems? I suppose this is the "new" ground Microsoft and Mindcraft are breaking together. :P

    logan

  • Furthermore, his Gamasutra profile says that he is a Microsoft Performance Engineer. Now isn't that damning?
  • by navindra (7571) on Tuesday April 27, 1999 @08:31AM (#1913637) Homepage
    This Will character has been traced even further [dejanews.com].
  • So what do we need to do to head off MS's ability to say:

    NT beat Linux by a factor of X with the help of Linus Torvalds himself.

    Linus has pretty good media coverage and he could use that. This is such a waste of his time. I think that the Salon article was excellent and should be seeded far and wide to the press. Anyone else have observations on that? Also, any ideas on where it would be best to send pointers would be appreciated. I think it's important that the news of their latest plot be aired *before* the results of the tests are released, after which time we'll just sound like we're whining. Perhaps being in the spotlight will make them less bold in publishing their figures, as well.
  • I forget the name of the security certification, but it's one the US Federal government requires systems that it buys to meet.

    Anyway, NT 3.51 met this certification only when NOT CONNECTTED TO A NETWORK. Only NT 3.51 meets it.

    Yet MS continues to sell NT 4.0 as meeting the security certification whose name I forget.

    They are one of, if not the, most blatently dishonest companies around.
  • I've met a few people who view Microsoft like that, but most people I've met are more cynical of MS.

    They see the constant deadlines being pushed back, MS not delivering on promises, MS getting caught astroturfing in the past, new MS technologies not working as they should, and lately very hyped MS only viri, etc.

    MS IS looked at as a safe choice only because we know that they will not be going out of business tommorow (hence: "You can't go wrong buying MS")

    MS is going down a slippery slope, W2K(NT5) will make or break the company, if past performance is any indication, NT5 will not live up to its hype.

  • Of course client server has been dying lately in favor of web/application server solutions.

    Sure this is a form of client/server, but the MS c/s server model was fat client, where you install a client program on 500 PCs, that take up 450megs on your PC for a "slim" install or 700megs for a full install, and require 64Meg ram to get by. This had the advantage of locking you into MS technology all around.

    I also fail to see how NT is an improvement over VMS. VMS at least was a multi-user OS, and I never saw a VAX go down for anything except a hardware failure.
  • You know it. They are panicking like HELL now. They are damned if they wait, and damned if they release now. They will release it soon, meaning it will be bloated, full of bugs (even more than the previous ones) and most importantly they must slash whole technologies that aren't finished like Intellimirror. And that was the major reason of the update in the first place.

    Well, if they had done multiuser correctly in the first place they wouldn't have this problem. But if you start growing your technology tree from a seed called Quick and Dirty OS, what can you expect. :-)

    Windows Y2K will be the end of Microsoft's OS dominance, you can count on it. (Still, I don't want them to go completely bankrupt. Age of Empires 2 and that new optical USB mouse look sweet!)
  • There's an old quote, attributed to Ghandi, and often cited in linux advocacy:

    First they ignore you. Then they laugh at you. Then they fight you. Then you win.
    These four stages of victory map nicely onto what's happened and is happening between Microsoft and Linux:

    First they ignore you. Microsoft did this for years, not really surprising anybody.

    Then they laugh at you This comes from from all levels, including Gates himself, and mostly takes the form of FUD. Do you really want an OS developed in some guy's garage? Is linux going to be here tomorrow? Linux has no roadmap. Can linux really be well-tested? I consider all this FUD to be a form of laughing at linux; it's really too indirect to be considered fighting.

    Then they fight you The Mindcrap Affair places us full-square in stage three. Microsoft is actively fighting, trying to bloody the penguin's nose (beak?). Granted, they're fighting dirty, but they are fighting. And get used to it; we're probably going to see a lot more of this from Microsoft.

    So why is this good news? Because it puts linux one step closer to stage four: Then you win. And it won't happen soon enough for me.

    --JT

  • Let [Red Hat|Caldera|SuSE|et al] buy the hardware for the servers and everybody bring your machine, we'll knock an NT Server out and benchmark a Linux Server. It shouldn't be THAT hard to do.
  • To conduct its study, Mindcraft compared Windows NT Server 4.0 and Red Hat Linux 5.2, upgraded to Linux 2.2.2 kernel. Both systems ran on a Dell PowerEdge 6300/400 server. Mindcraft equipped the Linux system with Samba 2.0.0 as its SMB file server and Apache 1.3.4 as its Web server. Windows NT Server 4.0 already came embedded with file and print and Web server capabilities. This class of system is what enterprise customers typically use for their enterprise servers.
    This seems to imply that Apache and Samba are not-included and cost money. More M$ FUD.
    xm@GeekMafia.dynip.com [http://GeekMafia.dynip.com/]
  • When I worked in an NT shop this summer, those MSCEs thought NT was the best thing since sliced bread. They laughed and couldn't believe it when I formatted my drive and installed Linux. They would believe this FUD without ever questing it.

    That is unless some mainstream mags like PC Week published a counter explaining it was bogus. I don't think they have...

    Anyone know?
    Romans 10:9-10 [gospelcom.net]
  • This summer I worked with some MCSEs. They think NT is the greatest thing in the world. When I told them about Linux and critized NT they gave me a book to read. The whole thing was written such that anyone who had not used it would sell their soul to it. It gave the whole background of how Microsoft hired the guys from Digital that made VAX. They had OS experience before so they could learn from their mistakes and make a new OS from the ground up. They talked about the client server based approach and how the kernel follows this design.. blah blah. And of course, they use the scallable, reliable, portable 'It runs on alpha ;)', buzzwords. In fact, those buzzwords are a lot of the NT cert exam. But to get back to the point, they believe it all. "98 is a peice of crap but not NT, here.. read this and you'll see" It's really really sad but sooooo very true.

    I do wish all Linuk was mkLinux though. I think the microkernel design is a cool way to do it even if having everything pipelined is a bit faster.


    Romans 10:9-10 [gospelcom.net]
  • The way M$ doc referes to client/server is the internals of the OS. For instance, the posix layer would pass infomation to the kernel layer.

    In this case, posix is client and kernel is server.

    Sorry, I forgot to specify before.
    Romans 10:9-10 [gospelcom.net]
  • The study is supposed to be directed towards IT professionals and assorted PHBs that are involved in setting future OS direction, coming up with scaling plans, hardware and software budgets for the next budget period, et cetera.

    These people are asking (or at least should be): what should I be running six months to a year from now in order to handle the predicted traffic and server load?

    Considering how quickly the semi-conductor industry is still moving, I think 1 GB quad-Xeon boxes are going to be more or less the standard new server box in a years time.

    Of course, Win NT scales to 4 processors so badly that Microsoft is artificially retarding the development of SMP hardware, so I could easily be wrong and we'll be stuck with 2-way SMP for longer.

    For example, there is almost no available Intel hardware bigger than 4-way SMP, even though handling more processors is well understood: Sun's entry-level(!) enterprise server E450 handles up to 8. There is nothing keeping the Intel hardware vendors from building bigger boxes besides the limitations of MS operating systems.
  • by Athos (11806)
    Wow. A relatively clue-filled news outlet. One that's not geek-centric and can talk to the outside world.

    A refreshing change from ZD!

    (CmdrTaco: can you get Andrew to do a piece or two for /. ? :) )
  • I get the idea that the MCSE's you worked with are pretty junior, and not represenative of most MS engineers.

    I've worked on-and-off with NT since 1994 (shit - 5 years!), and generally the MS technical community is very cynical about Microsoft's business practices and marketing, and especially suspect of MS's technical claims or benchmarks. Many of the folks I've worked with have been through the Microsoft PR gristmill with Novell, OS/2, and various midrange systems, and have seen it all before.

    While things like the Mindcraft study will impress the totally clueless PHBs and the Gee Whiz NT admins, no technical person with experience dealing with MS would give the thing a moments worth of thought. Even the ZD "PC Week" article essential was headlined "MS rigs benchmark".

    Microsoft's user loyalty is a mile wide and an inch deep - don't forget that when dealing with MS Zombies. When the Unix/Linux tide starts rising, they'll follow right along.
    --
  • Micrsoft so proud of the survey while everybody knows it's fake.

    Does everybody know it's a fake?? Microsoft is aiming this type of crap at senior management: People who have limited capacity for critical thinking and would rather have their homework done for them by "independent" reviews.

    One would hope that their technical experts would set them straight but unfortunately with senior management, once the mind is set, it's a helluva task to change their mind.
  • The reason no one replied to his message is that he gave *NO* information on what he was doing. Saying, I have a 4 XEON and I want to run a server is far from enough. Looking at the orginal posting, I did not see any discussion of kind of netcards, how much CGI, or even what kind of business they were doing. They just posted for the token, "We asked for help" don't blame us.
    The whole Mindcraft thing is a joke. Linux will never get a far shake from them with MS paying the bills.
  • The people who design planes are not the best ones to ask about flying them.

    It's all well and good for Linus and Alan Cox to help tune things for a benchmark. But actually, they are not the best people to do so. The best people to do so are those with domain experience -- that is, people who implement and manage large systems using Linux.

    And again, let me warn everyone: benchmarks are a moral hazard.

    --------
  • by BiGGO (15018) on Tuesday April 27, 1999 @08:43AM (#1913655) Homepage
    I can't believe the fuss.
    Micrsoft so proud of the survey while everybody knows it's fake.
    Saying that "NT is for mission critical applications" and is stable, is the stupidest thing ever.
    (I wouldnt trust NT to feed my cat, and I don't even have a cat)
    People know that they lie, and they become even less credible.

    I think they make a joke of themselves, and it's quite funny, too.
    In their struggle to survive Linux, they are being fools, and that will cost them costumers.

    People are starting to realize what Microsoft is about.
    Their crap bloatware is getting on people's nerves,
    and the legal tactics are now more understood by the public (thanks to the DoJ).
    But more of it, people are sick of being fed lies, and obviously bogus facts.

    We don't want fake videotapes and false promises.
    I'm glad to see Microsoft like that.
    I'm just waiting for the big crash.


    ---
  • I can believe that Micros~1 wants to believe this. They have this almost mystical ability to wish whatever they want into reality. Except that it looks like they went to the well one too many times and nobody is gonna buy it this time.

    Between the overt attempt to fake a "groundswell" of support, the bungled videos in front of the DoJ and now the "benchmarks" that "prove" Linux is inferior to NT (as shown by an "independent" testing lab, no less), the Emperor's new clothes are revealing what Microsoft *really* means...

  • This whole situation is good news for the Linux community. Microsoft is forced to respond to what WE are doing. They're running scared.

  • Wouldn't it be easy for Microsoft to spin off this so-called company and pretend it's independent

    I can verify that Mindcraft is not a Microsoft spinoff. They have been around for over 10 years. They are, however, essentially 'mercenaries'. I cannot say, however, that Microsoft hasn't purchased all or part of Mindcraft, although MS, being under scrutiny from the FTC (as well as the DOJ) would have to divulge any such investment. Their original purpose was doing POSIX compliance validation for OS vendors, however, they have also been doing benchmarking for a long time. At one time their primary customer was IBM (they provided benchmarking numbers for the RS/6000 in its early years).
  • I also fail to see how NT is an improvement over VMS.

    Of course not, it isn't. NT is a half-baked re-implementation of MicroVMS with the bloated Windows GUI sitting on top of it. It isn't an improvement over even old VAX VMS, let alone recent OpenVMS.

    VMS at least was a multi-user OS,

    NT could be if it wasn't strangulated by the Windows GUI code which is inherently rooted in a single user philosophy.

    Personally, I never even liked VMS. It was unecessarily crufty and complex and built from the large monolithic applications philosophy which I was never comfortable with.

    I think NT has all the worst aspects of VMS without any of the redeeming qualities. If NT didn't come from Microsoft, it would be dying even quicker than VMS is.

  • And test results like these help explain why Windows NT Server 4.0 has so much support

    Since you didn't state it in quite specific enough terms for the sarcastically challenged to grasp. Here is my read between Microsoft's lines:

    The fact that we (Microsoft) have to pay a company like Mindcraft to rig test results shows why NT is losing the war as a web server platform to *nix and Apache.

  • All these benchmarks have done for me is question the integrity of both Microsoft and Mindcraft.

    From now on I will take all claims made by Microsoft with a bigger grain of salt.

    Microsoft will survive this debacle, but I'm sure that Mindcraft will take a serious blow. With their integrity questioned, who will take their tests seriously? I don't think that they'll be getting too many customers now.

    Here is a suggestion for the Linux community: Someone should start a Linux hardware benchmarking and testing project. The role of this project would be to provide tips for tuning different configurations of computers for different purposes. Maybe they could get VAResearch to loan some hardware for the tests. The project team could challenge people at Microsoft to beat their results in head to head competition with impartial judges.
  • Actually, it was mentioned recently in some blurb about Linus' Comdex speech that his development machine is quad cpu (I don't recall if they mentioned what cpu in particular). But, in general, your point is well taken.
  • What is to prevent Mindcraft from using the advice they get in their write-up (we did this and this...), but not actually using it to tune the Linux server? Any study by this so called company should be suspect.
  • It is clear that this whole debacle is just another example of corporate propaganda. It makes you wonder just how many of the alleged "tests" that companies use to promote their products are truly independent tests, and how many are done by companies with the customer breathing down their necks.

    As an aside, I remember reading about how the tests were done with about 140 machines running Windoze creating the load on the server. For a truly fair test, I would like to see the tests repeated with 140 Linux machines. Even better would be 140 assorted machines with completely unknown operating system configurations: Windoze95, Windoze98, Linux from various distributions, Macs and so forth. After all, isn't that what happens in the real world?

    ---

    When you read anything on microsoft.com, remember:
    microsoft.com is the official organ of the Microsoft Corporation politburo
  • It may happen sooner than some think.

    I read in the business section of this morning's paper that a large investment firm (sorry, I forgot the name) is selling off all of its Microsoft stock, about $280 million worth. Apparently it figures that the stock is about peaked out, and there are better places to put the money.

  • by Mr Bill (21249)
    The problem is, that the server is only part of the cost. In order to test the server, you will also need 100-200 client machines + all the networking for these machines. This is where things get pricy. And to do a proper benchmark, asking people to SlashDot the machine is not an accurate measure of it's performance.
  • It worries me to read in the Salon article than Linus and Alan Cox are trying to give advice to Mindcraft for a second test

    It worries me because I think the Linux community did a very good job of refuting the ridiculous results of the test, but if they redo them and can claim that Linux lost even with the help of Linus and Alan Cox. . .

    But words will only take you so far. An MSCE could talk for hours about how great NT is, but that doesn't mean you're going to believe him.

    If Linus and Alan have a chance to help in redoing the tests (assuming the tests were done to their satisfaction), then I say go for it.

    It seems that maybe there is some fear that Linux may not "blow away" NT like it has been talked about. This wouldn't surprise me, given the oversized hardware the test is being done on. It is already known that Linux has a lot of room for growth in that market.

    It is not a bad thing to openly display the shortcomings of Linux. This will spark interest in overcoming these problems (and as the Salon article mentions, the ball is already rolling on this).

    But then again, maybe Linux will "blow NT" out of the water. I for one would like to see the results.

  • An even better test would be to post the address on /. Why use 140 machines when you can use 70,000. Just give us a time and a link and we'll see what it can really handle.
  • I'm pretty sure Linus's box is now a Quad-Xeon. Certainly it is Quad-x86 Intel something.
  • This is almost funny:

    "Microsoft was pleased but not surprised by Mindcraft's results concerning the excellent performance of Windows NT Server," said Ed Muth

    Well *of course* they're not suprised! This phony crap is so transparent. Next time they should have NT edge out linux by an order of magnitude, not just double or triple, then they could say they were "pleasantly surprised".

    Bunch of lying, cheating, conniving, manipulative turds. Makes me want to thow up.
  • by JEP (28735)
    From the MS webpage:


    "And test results like these help explain why Windows NT Server 4.0 has so much support."


    Ah, they speak the truth...

    --

  • WindowsNT Server 4.0 faster, more scalable than Linux as a file and Web server, tests show
    WindowsNT Server 4.0 significantly outperforms Linux in enterprise class systems
    Redmond, WA--After these tests, WindowsNT Server 4.0 is still at the head of the class.

    Microsoft SHOULD NOT be able to get away with such a cunning, blatent lie. There should be mass attention brought to this and expose these damn fools.

    I want to see front page headlines in say the NY times like "Microsoft are cheats"

    It's one thing to defend your OS, but its another to setup a scam like this.
  • Check out this account from a security consultant

    http://www.ntsecurity.net/scripts/loader.asp?iD= /news/022499-NT-Insecure.htm

    Not sure how biased or fair it is but I thought it
    was pretty interesting.
  • That is unless some mainstream mags like PC Week published a counter explaining it was bogus. I don't think they have...

    Not gonna happen, they get too much M$ advertising money. Not to mention the fact they own a NETWORK. Linux coverage(+)in the mainstream PC press is still sparse, although you can see media bubbles rising from the riverbed to the mainstream (/. --> Network TV)
  • Microsoft has never made such serious mistakes in so short of a period of time with the rest of its competitors.

    you weren't paying attention to the doj trial, were you? ;)
  • With our custom performance testing service, we work with you to define test goals. Then we put together the necessary tools and do the testing. We report the results back to you in a form that satisfies the test goals...

    Why not put together the funds to pay Mindcraft? Commission them to do a real study, publicly announce it, make sure that the hardware is the same and get the best people in both camps to tune their machines. It could be a win for all parties. Except one :)

    You may say that Mindcraft is biased, but if you are a paying customer, they have to live up to their promise and guarantees. Of course they could turn down the work, but then you would have the truest indication of their motives. Money should be the same color, no matter who's hands it is coming from.

  • Microsoft's EULA prevents you from publishing benchmark results on NT without their consent. This is not pure MS evil, either; I've noted that my Netscape license has the same clause. The intent is likely to keep incompetents from testing bogus benchmarks. Linux has no such license.

    However, if one simply did the Linux side, then pointed to the already published NT numbers for comparison, this may be doable.

    Better yet, don't use the Mindcraft guys at all. Get a machine with the same hardware setup as Mindcraft published, throw Linux on it, tune it, and watch it scream. You or I can't afford the hardware, but dedicated Linux vendors can (several of who advertise here). Better yet, they have motive to refute the results. And they can invite anyone to the tests to examine the setup, in case people complain about impartiality...

  • by remande (31154)
    You don't need optimal performance on the Linux end. Normal performance, non-pessimal performance, can outpace NT. And this is a good thing, because you couldn't get optimal Linux performance out of Mindcraft if they got tech support from Linus himself (which is why he's not buying in).

    Sure, you can get optimal performance by compiling from source. You can also shoot yourself in the foot quite badly. I think Mindcraft was aiming for the right big toe.

  • It worries me because I think the Linux community did a very good job of refuting the ridiculous results of the test, but if they redo them and can claim that Linux lost even with the help of Linus and Alan Cox

    Don't worry. If/when they complete the "test" and publish it, naming Linus and Alan as "helpers", the first thing the news media is going to do is to call Linus and Alan to get a comment about it and thereby get the real story. In fact, having Linus and Alan involved in this actually helps us prove the point that MindCraft is not interested in obtaining decent Linux performance.

  • Mindcraft still have their report as the top link on their web site (www.mindcraft.com). This is AFTER MS admitted have publicly admitted it was a sham. I have sent 3 polite email so far asking them to do the decent thing - surprise! - no action, no reply from them.
    Anyone else want to give them a nudge?
  • So I go and have a quick read of the MS press release cum article and decide I want to send them a polite note telling someone somewhere within the MS organization what I think about this little piece. When I choose the one button on the page which appears it might be a way to communicate, I get a pre-determined choice of options, not one of which has anything to do with replying to the article.

    So, I look around a bit and click on a few items looking for a way to contact _anyone_ with a comment about the story and came up with the big zero: Nada, nothing, no way to say "Hi Microsoft, your story on the Mindcraft debacle has a few problems." This is typical MS in that MS doesn't want to hear from MS customers because MS is too busy trying to dictate what said customer should be doing and thinking -- and totally missing the boat all the while.

    Justice will be served when MS turns around one day and finds they are missing many millions of customers. Count me in that rapidly growing group.

    1 1/2 years of livin' and lovin' Linux!

  • Microsoft obviously does not practice what they preach.
    Just look at what their famed hotmail runs on -- netcraft.com [netcraft.com] speaks the truth!
    Microsoft is going to pay dearly in the end. Anyone with some sense has seen through this FUD already. Alot of people out there are in bed with Microsoft and take it's lies for gospel. They won't believe anything works but Microsoft and they are doomed for a hell working in help desks for eternity.
    Once the new study is run by Mindcraft with the full support of Linux users, Microsoft will be exposed for the fraud they are. Alot of the damage has been done already. Linux is still small and hopefully people realize that when they read Microsoft's news releases.
  • If you read their performance page, it says that the various systems are at "peak performance"
    So, by publishing this report as it was they are *certifing* that linux was running at peak performance
    And, several of the changes they made, appeared to even reduce the out of box performance
    So, are they *certifing* something that is obviously (and knowingly) false?
  • Surprises me that it is Linus who is going to help. Did they only agree if it was Linus himself? If that's the case, do they also require BGate$ to be there?

    Huh, got a much better idea, less do a benchmark and let Linus and BGate$ setup their own machine (same hardware of course).
    THEN run some tests. :o)

    Breace.
  • Should've finished up the article. It's a shame that Mindcraft is refusing to do a proper test. Apparently the truth will hurt that they rig results. Pity.

    Jack
  • The purpose of these tests is to please the customer who commissions them.

    I think that pretty much sums up the whole situation. And here Microsoft is trying to tout this result as something valid when the whole Linux community is up in arms over it, the press is outright calling the results a sham, and even Mindcraft is now wanting to redo the tests, most likely to get some respect back. I hate to say it (actually I LOVE to say it) but Microsoft is again making themselves look stupid by demeaning Linux. Hell, I just started running Linux full time after playing around with it for a year, and I have to say I've never been happier with speed, stability, and performance. Let real world examples show Linux is better than NT.

    Jack

  • Think again. RAID5 is _much_ slower then
    RAID1 on writes because it must recompute parity info on each write which requires several additional read and write operations. In some cases its read performance may be much slower then RAID1. For example many small files, opening file
    requires modification of its access time which is a write operation. You can observer this behaviour
    with INN or sendmail with large number of users.

    RAID1 is actually faster then RAID0 or RAID5 for reads because it has twice the number of disks and with proper implementation _all_ disks are used to
    fetch data.

  • Bullshit. Each disk added to RAID0/RAID1 speeds
    up writes. For RAID5 it may not be the case
    because each additional disk increases number
    of operations to recalculate the parity.

  • I don't think M$ support would help much, but at
    least it would be possible to find someone you
    could pay to set up a web server and tune it on
    NT. Linux web consultants are still hard(er) to
    come by.


    How much did Mindcraft pay (or how much would
    anyone who wanted the same level of support
    have to pay.) How would this compare with
    chartering a 747 to pick up some Linux, Samba
    and Apache people?

  • How much does a license for NT cost now? A little
    more than a Linux CD, I guess. The difference in

    Many, many times more, in the case of file
    serving. The more clients you want to handle the
    more you pay. (Somehow I doubt the "license
    manager" does any performance tuning...)

    software should go to the hardware side, if we
    want to compare configurations that cost the
    same overall.

    You also need to factor in chargable support.

    Anyway if you'd did this you'd be pitting this
    poor little SMP box against a huge cluster of
    Linux machines :)

  • They also ran apache out of inetd.


    Possibly with tcp wrappers on as well, which adds
    in additional DNS lookups and logging.

    Also the way they had IIS configured to log in
    large chunks is unrealistic for any real world
    situation.
  • Lunis will never have a chance to touch the machine in question. He won't be allowed within 50 feet. Imagine, here is linus setting up his server. He sees that SMP isn't working so great, so he hacks out a quick fix and patches the code. Presto, linux is kicking ass. Mindcraft whines, "Hey, that's not fair, you can't just change the kernel like that! That patch isn't in the official distribution!" And linus just grins and say, "It is now."

    When when they undertsand the concept. The number one best super-duper cool reason to choose linux over NT isn't performance (not that it wouldn't beat the shit out of NT) but it's the code. You have all the info right there at your fingertips to crunch out your own perfect server, and you never have to wait for someone to come along and stick a bottle in your mouth.
  • The sad thing about this whole thing is that M$ will ALWAYS be believed. I know that most NT sysadmins think that M$ is the absolute truth, and if it comes from them, they better believe it. M$ trumpets that NT is better than linux, and those die-hard NT sysadmin's will believe it (and won't bother trying linux out themselves). Those same die-hards won't believe thousands and thousands of real-world, pro-linux testimony, but yet they'll believe one company... and a lying one at that. Crazy, isn't it?



    Todd Gearhart
  • I was wrong in lumping together all NT sysadmins. Some of these guys (and gals too) are very good people how just have no choice as to which OS they use.


    Todd Gearhart
  • Money + Americans = Stupidity.

    Take note of our hypocritical society

    ___________________
    Buy [linuxppc.org] (I mean download) linux today!
    ___________________
    Learn [esperanto.org] something!

If you are good, you will be assigned all the work. If you are real good, you will get out of it.

Working...