ZD Critiques Mindcraft Benchmarks 141
SFraser
sent us a link to a pretty decent critique of Mindcraft's
benchmark which claimed NT was faster.
Monday's story about the
tests here on
Slashdot about the report was one of the most active
Slashdot stories ever.
Performance analysis toolkit. (Score:1)
along the lines of SymbEL/Virtual Adrian? For
those who havent seen these, they're free (and
open-source, mostly) kits for analysing performance problems on Solaris, and suggesting
fixes (mainly kernel parameter tweaks). See:
http://www.sunworld.com/swol-02-1999/swol-02-pe
I do mean 'analysis' not 'collection' like
sar etc...
I only ask 'cos I've not tried tuning Linux
(though I have done it on Solaris, obviously)
and searching on freshmeat for 'performance',
'tune' or 'tuning', only 'hdparm' seems relevant.
SymbEL is pretty nice, though obviously tied
to the Solaris kernel. A framework like this
for plugging in perfomance advice for servers,
in general, would be a mighty fine contribution
to the Linux cause.
- Baz
We NEED a Linux Tuning website... ON THE CASE! (Score:1)
A pre-alpha can be found at http://www.upsu.plym.ac.uk/~betty/LinuxEnterprise
(please don't all hit unless you have something to contribute, until I have mirror at
http://stacy.flwireless.net/atrevena/LinuxEnter
It is a bit sparse, but it is started and I will be happy to fold it into any other relevent project if it falls behind - I will have a US mirror for it in a couple of days but in the meantime -please be gentle as it.
Submissions to betty@area51.upsu.plym.ac.uk
I was also thinking of starting a Issues Database, using MySQL and mod_perl. It could of course be distributed according to Distie, App, Hardware, etc but with a round-robin dist front end it would be really handy and def. give the majors something to worry about without losing the hacker way.
Aaron (TheJackal - wtf is my cookie? wtf is my password?!)
Just one obvious conclusion: (Score:1)
When you are tunning a system and you are changing
a configuration flag, you usualy try to measure the impact of every change you do.
For instance, if you were going to set "WideLinks"
flag to a different value, you would benchmark the
system before changing the flag and do another
benchmark after changing it.
Only this way, you would know what was the impact of setting each specific flag.
I'm just a simple computer user. However, these persons are professionals runnig a test lab. They should know exactly what they were doing and
work strictly using scientific procedures.
If they were doing a serious benchmark, they
should have measured the impact of every tunnig flag.
After seeing that changing a specific flag would lower performance, they would set the flag again to the defualt value.
However, they changed a lot of configuration
flags that lowered the performance of the Linux
system. Didn't they noticed the slowdown efect ?
I don't think so.
If they used only "default settings", then I would
beleive that they didn't tune well the Linux system because they weren't enoungh competent.
However, they seem to have changed a lot of
default setting and, each time, the results
were worst than the defualts.
How can this be possible ?
Guess what was my Conclusion !
Mindcraft's #1 error (Score:2)
Show me a Benchmark w/NT 3 times faster than Linux (Score:2)
is the result of a poorly tuned Linux system ?
Personaly, I beleive these mis-configurations were deliberately done, by people that knew exactly
what they were doing.
They even choosed the right hardware (4 ethernet cards), because they were aware that Linux would try to route all the packets trough a single network card.
They don't even show the routing configuration and the subnets used in the test.
Was Windows NT using TCP/IP or NetBEUI ?
This way, this benchmark only proofs that NT with
4 network cards is faster than Linux configured
to use a single Network card.
This might be the reason why the Linux system has peak performance at nearly 100 Mbits/s: the limit for a single network card.
From the list of processes running before the tests were done, we can see that there was no "httpd" and no "smbd"/"nmbd".
This leads to the conclusion that both services were launched by inetd.
This is a very very slow option: each time the server receives a new connection the system will launch a new process.
The defult setting of red hat Linux for apache
and samba are standalone servers: this is much much faster.
Using inetd will realy realy HURT Linux performance.
The settings in the apache flags "KeepAlive" and "MinSpareServers" are consistent with the fact of
having apache and samba launched by inetd.
I just cannot call this a tunning. These settings are much worst than the default settings for both services.
The WideLinks flag was also a very nasty setting: Windows users cannot create Links and this falg wasn't necessary. It you want safety you can chroot the service. Even if you have several shares, you can mount all the shares in the same subdirectory tree and use "chroot" to that subdir.
Probably without this misconfigurations, NT wasn't
even fast enough ( 4 network cards versus 1 wasn't enough).
Finally, I don't think that this poor results have anything to do with kernel SMP performance.
This is just what I think.
An. Coward.
Does Mindcraft have a bias? Novell thinks so (Score:3)
Re: SMP support in kernel 2.0.36??? (Score:1)
Re: ... and on a silver platter! (Score:1)
It's more than just him... (Score:1)
ttyl
Farrell
Interesting point (Score:1)
I can't say more along those lines, but:
My first instinct, upon seeing Mindcraft's report, was "Wow, that Dell sucks! I wonder how much they'd charge to benchmark our Linux Hardware Solutions quad-Xeon against that box? We'd SLAUGHTER Dell!"
But then the realization struck: If these people showed such a lack of professionalism as they did in this report, would we WANT to pay them money? I.e., why in the world would we pay people who have already demonstrated that they are a laughingstock by releasing a report such as this which can be viewed, at best, as unprofessional, and at worst, as an amateurish attempt at dissing Linux?
Needless to say, any testing done in the future on our behalf will be done by someone OTHER than Mindcraft.
-- Eric
DH Brown was basically accurate (Score:1)
My only real complaint is that they said NT is enterprise-ready, when it so obviously isn't!
On the other hand, Mindcraft "study" was either the most inept piece of work that I ever saw, or a deliberate hack job. (And if it's a deliberate hack job, it's the most inept hack job I've ever seen!). The best thing that Mindcraft could do to salvage their reputation would be to withdraw the report "for further study" until its deficiencies are resolved, but I frankly suspect it'll be a cold day in hell first -- they've been paid for it already.
Of course, they could surprise me!
-- Eric
Requests for tuning support (Score:1)
ever tried ms knowledgebase?
Fixed link (Score:2)
-Ben
Corollary Links (Score:2)
Here are some other responses to said report:
http://lwn.net/1999/features/MindCraft1.0.phtml [lwn.net] -- Linux Weekly News [lwn.net]
http://www.linux-hw.com/~eric/mindcraft.html [linux-hw.com] -- Linux Hardware Solutions [linux-hw.com]
-Ben
Wasn't SMP support already in kernel 2.0.36??? (Score:1)
acknowledge that SMP support is *not* a new feature
in kernel 2.2.x -- if I recall SMP support was
part of kernel 2.0.36 on the Red Hat CD -- you
just need to install the kernel sources from the
CD and recompile.
Good point! (Score:1)
has unlimited assistance and total attention
from Microsoft -- certainly not the kind of
attention a typical NT admin would get from
Microsoft.
...and still they didn't bother hiring a Linux
guru to help them -- so in that respect Linux
did pretty good considering it was not
configured well *and* was setup by newbies.
I don't get it -- that's the same article (Score:1)
of the ZDNet article on the excite.com web site.
.and here's a short list spec for such a web site (Score:2)
2. Some typical Apache httpd.conf settings for
enterprise web sites.
3. A detailed list of which Apache modules can
be disabled on a plain vanilla web site
4. Ethernet driver recompile and tuning guides.
5. Which network services you can disable on
a plain vanilla Linux web server
6. Web server tuning tips regarding file I/O
and how NFS, RAID, SCSI, and multiple disks
affect file I/O performance.
7. Real world perfromance testimonials.
8. Slashdt style Q&A forums
9. Ethernet and network topology advice
10. Explanation of DNS tricks such as round-robin
and gateway switching.
This was needed (Score:1)
The cat has lost it's tail. (Score:1)
Also, I don't see how this could possibly be unfair to NT. When NT4SP1 was released, hardware like this didn't exist at all. I (later, years after it came out) installed it on a P133, with 32MB of RAM, and a 2GB HD, roughly half of the system you're talking about. Although it was a little sluggish, (and far slower than DOS or Linux on the same machine) it worked, and it wasn't unbearably slow. It was surprisingly bloated in its memory requirements compared to either DOS or Linux, but it still didn't have to swap.
When they first were releasing NT, the standard testing practice was to get the fastest 486 you could find, load it up with RAM, and... see how many copies of solitare the system could handle. I think Microsoft was relying completely on Moore's law here...
Now, one would think that NT4SP2, 3, and 4, as the name "Service Pack" suggests, would be minor, free upgrades and bugfixes? Otherwise, they would have released an NT version 4.1 or something? Nope. I'm currently running NT4SP3+ActiveDesktop on that same machine, and it's *incredibly* slow. The RAM requirements have almost doubled. However, the system requirements for NT 4.0 haven't changed, have they?
If you're going to blame anyone for a Server OS not running well on that hardware, blame Microsoft. They aren't obeying their own released specs, and they aren't serving their users. If NT4SP1 was slower than it needed to be, then why aren't later versions faster? Who would pay for an upgrade that doesn't fix the real problems? Maybe that's what "Service Pack" means.
Maybe that ZD article should have been entitled "Linux kernel upgrades actually add features *and* increase performance, unlike NT upgrades." Personally, I prefer the title "Windows turns fast computers into toasters that crash and burn toast", but maybe that's just me.
Regedit is easier? (Score:1)
Regedit is easier? (Score:1)
I've found the same sort of resources without much effort all over the net, including at apache's web page itself.
And as far as the microsoft knowledgebase, I DO use it myself, but it doesn't simply return what your looking for.. It returns several hundred possibilities, which you must then 'sift' thru. nearly the same queries can be run against www.hotbot.com to get the results..
Requests for tuning support (Score:1)
Minimum Hardware to equal Quad Xeon with NT (Score:1)
If a single or dual PII with a fast SCSI disk and Linux could equal or beat the performance of the Quad NT machine then both Micro$oft and Mindcraft would be embarassed.
tuning howto -- volunteers? (Score:1)
Requests for tuning support (Score:1)
The items that mindcraft used for tuning NT are fairly well established, either from NT documents or documents relating to the NICs, etc.
I don't know as if I like RedHat's response, that if the request had been put forth to their PR department they would have responded. What if I'm an enduser and just looking for performance tuning information, rather than some big testing company?
Does RedHat only offer performance tuning advice to marketing reps?
That'd be kind of like Ford saying "Well if we'd know they were from Car and Driver, we would have fixed that transmission on their demo unit, instead of stonewalling them."
The cat has lost it's tail. (Score:1)
Nobody in their right mind would install as a server a Pentium II 266, 64 Meg RAM with a 4 gig IDE drive. This is the hardware that ZD used in their test.
This is really unfair to NT, as 64 Megs is the bare minimum at which you can really run IIS. Try the test again with 128 megs, or 256 megs and see what happens. Then try it with SCSI instead of IDE, etc. Why not use a Proliant 800 instead of a desktop machine?
So as ZD said in this article and Linux scaling up, their other article should have been titled "NT Server doesn't scale down as well as Linux".
It's him again. The Misfit :) (Score:1)
Who cares (Score:1)
Of course, the other question is-
Even if in a fair fight on a multi-processor machine NT kicks linux's rear to cleveland and back. The fact remains that Linux has gone from an unknown to a contender in... a year? A year and a half tops?
The fact also remains that NT is backed by Microsoft and Linux is backed by the rest of us. We're reading articles that are *comparing* NT to Linux and stating that it is a fairly close race in many common instances and even victorious for Linux in the most common of those races. Viva Linux!
Does Mindcraft have a bias? Novell thinks so (Score:1)
tuning howto -- volunteers? (Score:2)
I have some bandwidth (not a ton, though... like a few hundred MB a day... not slashdot sized :-) to spare. If people would like a resource like this, just email me at sujal@worldnet.att.net and/or sujal@sujal.net with suggestions, comments, etc.
If someone else is willing to do this, or has already started, please email me so I can help them rather than doubling up effort. Thanks
Sujal
Tuning Linux, really? (Score:2)
The only answer I came up with is that Linux actually requires little tuning. Probably 95% of the Linux boxes out there need virtually no tuning. It runs so well out of the box that tuning it is hardly necassary.
Microsoft, on the other hand, has basically acknowledged that out of the box, NT runs like crap. They've provided lots of information on how to make it run better. I could only see them doing this if people are unhappy with its performance.
I know I'm more than happy with the performance of my untuned Linux boxes and I'm not sure that spending lots of time tuning them would really make a big difference.
** Martin
Dejanews is *my* Knowledge Base (Score:1)
I often search for the exact error message. 9 out of 10 times it's there, with several Re: replies...
It could have been better... (Score:1)
That said, I was a tad disappointed with the ZD article. It had no hard figures to prove the idea that Linux is faster. ZD recently did a benchmark test of some sort comparing NT and Linux, but I can't remember if it was web and file serving.
What I'd really love to see is for Linus to rebut the report at Comdex. More specifically I'd like him to do explain in understandable detail (perhaps referring to a more informative url to move things along better) why Mindcraft's Linux "tuning" was inappropriate. I'd like him to offer up a few bits of documentation explaining the Linux tuning process to deflect the rightly earned lack of docs critique. And before doing the new benchmark results, explain any new patches that help improve the system that were brought to light thanks to the benchmark - it wouldn't hurt to mention that those patches were coded, tested, and put in place within a week.
Finally, of course, benchmark results demonstrating that Linux bets out NT in this newly conducted test. That would be quite nice, and save Linus the effort of coming up with a speech *and* a topic.
DH Brown (like Mindcraft) says NT beats Linux too. (Score:1)
Of course, I'm not a hardcore kernel hacker, so I don't know whether what DH Brown says or not is true, but it all sounded factual according to articles I have read previously. I think Linux Journal ran an article on a talk with Intel, Linus, VA Research and RedHat, where Intel points out the lack of 36bit memory addresses and offers to help.
Wow zd-net doing some anti-fud for once. (Score:2)
I heard a interesting thing, about technology writers having a more jaundiced eye towards Microsoft after some of the trial shenanigans. It maybe that this is a sign, that technology as a industry, is groing up a bit.
Requests for tuning support (Score:2)
Some of the items that have been bandied about is stuff that is obscure and difficult to figure out. For example, I wouldn't expect even a fairly skilled Linux user to know that the 2.2.2 kernel had problems with TCP and windows clients.
However, anybody even vaguely familiar with the linux versioning system should know always to get the lastest version of the stable kernel. No one getting paid to administer a Linux machine should look at a list of kernels 2.2.1 - 2.2.5 and decide to download 2.2.2 !
Moreover, some of the mistakes they made (compiling samba with -O instead of -O2, setting the number of servers to 10 initially and 1 spare) show a lack of even basic Unix and apache administration skills. While it may not be fair to expect them to nail every possible optimization, there's no excuse for neglecting basic stuff like getting the latest stable kernel and telling apache to start enough servers. Even if this sort of information does require a fair amount of Linux knowledge, who in their right mind would pay over $25,000 for a machine and not even hire an admin with a basic knowledge of its operation ?
Even if they didn't hire a competent Linux admin, I'd pit Linux's performance under a newbie admin against NT's performance under a newbie admin. The ZDNet article claims that performance tuning information for apache is much more difficult to find than that for IIS. But, if you go to the apache web site and click on the big link named 'Server Documentation', you'll find links to both general and platform specific performance tips. The section on general performance tips includes *gasp* a couple of paragraphs on the importance of setting the StartServers and MinSpareServers directives appropriately. In contrast, the Microsoft site requires you to navigate through a maze of links to try to find information about IIS. I frankly gave up trying to count the number of links until I could get server performance tips for IIS after ten minutes of wading through MS's marketing classifications (am I an enterprise customer or an IT professional ? which support option do I need ? Why don't many of the forms work in Netscape? etc. etc.). The reason that the ZDNet author believes that the IIS information is easy to find is because he's used to using that information, just like the authors of the study found NT easy to tune b/c they were used to tuning it.
However, if such a newbie-Linux-vs.-newbie-NT-on-an-enterprise-class- machine-for-loads-greater-than-those-han dled-by-Yahoo study were done, it would be quickly dismissed as patently ridiculous. Mindcraft was smarter than that, so they actually pretended to tune the Linux box, but that doesn't change the fact that they were testing the performance of of an enterprise class system and ridiculously high loads under a clueless admin.
Lastly, I challenge you to call up the Microsoft Tech Support line, tell them you're running a perfomance benchmark on NT and would like some tuning help. I seriously doubt you'd get anything even resembling a helpful response !
Wow zd-net doing some anti-fud for once. (Score:1)
of their benchmarking suite, probably most of
it is that it's a story they feel they ought
to run.
Widelinks = False is not the default (Score:1)
One of these three possible scenarios must be true, and all of them mean that they're not playing fair.
We NEED a Linux Tuning website... (Score:1)
I may be wrong about this, but I haven't found one.
What this site should do, is be a repository for all kinds of Linux tuning - Kernel tuning, Apache Tuning, Oracle tuning - everything that can be tuned should be documented here!
search engines vs. MS Knowledge Base (Score:1)
ZDNet not so bad after all... (Score:1)
Not that I really pay attention to ZD stuff anymore... but I really appreciate their remarks on this test.
It is in the true spirit of journalism to shed light on the truth no matter how it affects your own interests.
ZDNet not so bad after all... (Score:1)
Good ZDnet article, a couple of pointers (Score:1)
Red Hat Linux is not available with the 2.2 kernel. First of all, you CAN download RH 5.9, sure it is development, but it is a helluvalot more stable than NT w/o service packs (even with). But that isn't my point. There ARE 2.2 distros available now. I like RH, I use RH, but for goodness sake it isn't now nor will it ever be the only Linux distro, thankfully. Sticking point
Linux documentation hard to find wheras MS is easy to find. Okay, linux documentation varies a lot and sometimes is hard to find, but have any of you really ever tried to find something on MS's juggernaut of a web site? I was an NT guy about a year ago, but trying to find any REAL answers from their website only resulted in my getting lost in it and my seeing way to much propaganda.
If it's not PRO-Linux, it must be fud! (Score:1)
Instead of blindly attacking anything that isn't PRO-Linux, try to open your eyes a little, stop your screaming and actually look at what's going on. It worries me that the "linux community" is becoming more and more a bunch of guys who pat eachother on the backs and fanatically reject anything that isn't PRO-LINUX. Do you really think this is a good direction?
Also, I would like the suggest that the moderators go re-read the Moderation guidelines. Especially the part about not downgrading a post simply because you don't agree with the viewpoint. Go look over the mindcraft thread and make note that most of the -1 posts are well-written, but take the side of Mindcraft. I thought opensource was about freedom of speech and against censorship?
Don't be a bunch of hypocrites and censor anything that is not PRO-LINUX.
ZDNet not so bad after all... (Score:1)
There is nothing that prevents MS untruths (Score:1)
This is it guys and gals. The guns are out and they are aimed at Linux. The last OS that these guns were pointed at was OS/2 and we know where that OS landed. Can you say niche? How does this get stopped? There is no law against this kind of untruth, they would just say there was a misunderstanding or a bug and it will be fixed at a later time. The DAMAGE is done. A later retraction means little.
I'm sending email to James Love to see what they think about this kind of public deception. A group with some pull needs to address this, NOW.
mailto://love@cptech.org
Locutus
of OS/2 and Netware (Score:1)
Micros~1 has $$$ to buy whom every they please and MindCrap is just the most current one.
( I was told that MindCrap may have been used my Micros~1 to "prove" RealNetworks was at fault for breaking the RealPlayer on Windows) Says something if it is true, doesn't it?
Penguins need missile launchers since snowballs don't cut it now that war has been declared.
Locutus
Linux: Defeated by MS at SMP! (Score:1)
Someone might say,"Oops I guess we shouldn't have used those settings. We wouldn't do that again if we were to run the test tomorrow......"
Sorry, but you must be a newbie because Microsoft is involved and nothing logical and ethical matters. FUD is the name of the game. FUD kicked OS/2's butt, it kicked the Macs butt, and it damaged Netware pretty good too. This 'test' was to cast DOUBT that Linux can do 'the job'. Bill Gates is spreading UNCERTAINTY everywhere he speaks. FEAR, that is a tough one because it is very difficult to notice someone cowarding in a darkened corner. So a project is killed misteriously by upper management. Who really can say why? Look what Microsoft did to Intel and its Media Labs, tried with Apples QuickTime. Linux is Microsofts threat and history has shown what they do to threats. Linux does need some work but obvious de-tuning of a test system and proclaiming to the world about its defeat is classic Microsoft style.
Locutus
Good to see (Score:1)
Now, the article does suggest that Linux has a ways to go with SMP/RAID support, but it does not suggest that it cannot rise to this. I think this benchmark is useful, not as a true representation of the facts, but it does show the community what it has yet to accomplish, and that the goal is well within reach.
tuning howto -- volunteers? (Score:1)
The code / db is still around so if anyone has a server to spare with some decent bandwidth, let's go.
The database is still in its infancy but it's a starting point for a project.
Any takers? Email me at simond@foxlink.net.
Necessity is the mother of all innovation. (Score:1)
Simply put, NT on a Pentium 200 with 64MB performs horrendously compared to Linux. Computer manufacturers realized this a long time ago, which is why quad-Xeon systems are being pushed like wildfire for any serious applications.
So, just in sheer man-hours of development, I bet that NT has more than 4 times the amount of work invested in the required support than Linux does. We can't really blame Linux for this one. I'm quite sure that if the [Linux] developers could all afford quad-Xeons like Microsoft can, the race would be much different.
Put simply, I'm not at all suprised about these test results. If Mindcraft had any serious grasp of Linux, they would have known that it was at a disadvantage before the tests were run.
Funny that they didn't do the "Bang for the Buck" comparison. A quad-processor NT Server license ain't cheap.
But it states NT wins... (Score:1)
What I say in the paragraphs, preceding the lines quoted, boils down to that Linux SMP/RAID isn't as mature as NT SMP/RAID. Anyone disagree? I didn't think so.
As I point out earlier in the article, there's not even a gold driver for the in-test RAID controller. Given that Linux, for this platform, is running in a beta configuration, no matter how well tuned, it would indeed be a 'bear' for Linux to prevail even with tuning. Even so, though, I state, based on my testing and a lot of back and forth to other writers, benchmark writers and Linux experts at PC Week, S@R, Samba and ZD Benchmarks folks, that "Linux's numbers would have been much closer to NT, if not HIGHER, had Linux been as well tuned as NT."
That's not winning in my book.
Steven, Senior Technology Editor, Sm@rt Reseller
PS. Were high-end scaling the be-all and end-all of what you needed in the real-world, everyone should be running AIX. It's not and people aren't.
Wasn't SMP support already in kernel 2.0.36??? (Score:1)
From where I sit, SMP really only becomes a practical option with 2.2. There's lot of work still to be done, but it's work's being done by a lot of good Linux kernel folks.
Steven, Senior Technology Editor, Sm@rt Reseller
They didn't say that M$ paid for it at first (Score:1)
Steven, Senior Technology Editor, Sm@rt Reseller
The cat has lost it's tail. (Score:2)
When Microsoft stops claiming that that's more than enough machine for a NT 4 server, we'll quit reviewing NT on similar platforms.
Another reason for our choice of platforms is that while we all lust in our heart for quad-pentiums with a gig of memory to call our own, the simple truth is most of us, and most businesses, can't afford them. Yes, CPUs and memory are cheaper than ever, but businesses don't replace their legacy equipment all that often. Our studies show us that a plurality of local LAN business servers out there are less well equipped than our test machines. So it is that one reason we test on low-end systems is that that's what many of our readers still have to deal with it.
Again, though, if MS just said you need at least X and Y to run NT, we'd use it. It wouldn't be fair to do otherwise. But, they don't, the users don't, so we don't.
At least with 2000, they're being 'somewhat' more realistic--128MBs and 300MHz minimums. I quote 'somewhat' because I suspect that those minimums will prove to be bare-bone minimums.
Steven, Senior Technology Editor, Sm@rt Reseller
We NEED a Linux Tuning website... (Score:1)
Anyone been dying to set up a Linux web sight? I'm sure if the person who sets it up makes themself available for email suggestions and has a web based message board, they could collect most of their needed information rather effortlessly. Either that or Caldera or Red Hat need to set up a web sight.
This might also make a good topic of an O'Reilly book. It could have a song bird on the front.
Any way, my $.02 minus sales tax. I hate Apr 15th.
Way to go. (Score:1)
Please mention Sys V IPC tuning too if you could. I'm working on a project now which would require a bit of IPC tuning in the kernel and for me it is a sinch, but for others that will evetually have to install the system themselves, this would be a stumbling block. Many newbies do not yet understand the art of recompiling their kernel, and don't unless they absolutely have to. Even some seasoned Unix users shiver when you tell them to recompile their kernel. It's sights like this which I think will help people adjust to the administrative culture shock.
linux tweak site (Score:1)
im sure that would get lots of hits.
Comparitive Testing Needs Funding (Score:1)
"Linux support lacking" (Score:1)
As I recall from the original story discussion, there was a single posting, from a private individual, requesting information. That request never even mentioned the fact that the comparison was to result in publication. Had they made the disclaimer, and admitted to being an 'independent' test lab - the Linux community would have surely innundated them with helpful suggestions.
In effect, they shouted a question off a mountaintop and never stuck around long enough to hear the echo.
I'm sure that the querry they submitted to RedHat was equally low-key; probably to accentuate the documentation deficit that Linux suffers.
I imagine is went something like this:
Mindcraft: "Hi, I'm Joe Shmoe. I just bought Linux and I want to fine tune it."
RedHat: "We'd be glad to help, but you need to be more specific."
Mindcraft: "Well, you're not at all helpful, are you?" CLICK!
Gee, RedHat wasn't very helpful, were they?
"Linux support lacking" (Score:1)
As would I. What would it take?
I've written minimal documentation for some of my programming efforts, but it was corporate work, and so the docs were not much more readable than the code by the time the reviewers got done.
As for making something searchable, context sensitive, intuitive... Well, I know nothing about natural language querry processing, but I do like how helpful the Office 97 help system is. It's not concise, and it assumes ignorance, but it is pretty smart.
Who in freecode land knows how to structure such an effort?
Some technical details posted to ZDNet (Score:1)
Finally, a decent article from ZDNet (Score:1)
Actually, they DO mention this. At the very beginning, in bold type even, they call this a "Microsoft Commisioned Study".
... and on a silver platter! (Score:1)
It doesn't matter who pays... (Score:2)
I'm all for applying critical thinking and considering the motivation of the speaker. I'm against automatically assuming a claim is false just because someone we dislike made it. I've seen plenty of usenet flamewars in which entirely true study findings were discounted solely on the basis of who paid for them. That's just closed-minded.
On the other hand, specific point-by-point rebuttals are right-on. Producing evidence that refutes the claim based on better science in golden.
I guess it's just easier to be a mindless naysayer and pretend to be a sophisticated skeptic than it is to approach disagreeable ideas with an open mind and use logic and rationality to pick them apart, and risk having to admit being wrong.
fud (Score:1)
While I agree that linux users do sometimes overreact, I think that almost everything that says linux don't work is also a kind of reaction - 'If it doesn't fit what I currently believe as right it is wrong and I should by all means try to destroy it, but after the Media says it's right I will rethink and say to all, "forget what I said before"'.
It doesn't matter who pays... (Score:1)
It is standard practice in evaluating reports to make note of the entity responsible for commisioning it in order to consider possible bias.
... and on a silver platter! (Score:1)
tuning howto (Score:1)
While code yells at you if you aren't syntactically correct, documentation doesn't, or at least not as fast. Anyone who reads code to modify it, will have a much greater chance of understanding it, even if poorly written/documented, than a newbie who is trying to learn something (a howto reader).
Regedit is easier? (Score:1)
But to answer your question,
1. Windows NT Resource Kit
2. Microsoft Technet
3. Microsoft Knowledge Base
in that order. Just about everything I know about NT has come from those sources. When I read a book about NT, I find it is generally a dumbed-down how-to derived from those sources.
As an example to address your question (and not a great one, since it's for an NT client - Win 95 - and not NT itself), I remember the first time I had occasion to use those resources, as a phone tech in 1995. Somebody couldn't log in to their NT server with Windows 95. I searched TechNet for the error message they were getting and found a registry setting that would cause the problem. At the time, I knew NOTHING about networking - all I knew was how to do boolean searches against an MS database.
So yes, I agree that it's a lot easier for the inexperienced (and the stupid) to optimize Windows NT than Linux. Kind of like saying it's easier for a 10-year-old to make a paper airplane than a Boeing 777
A Bad Reputation Is A Hard Thing To Change (Score:1)
The MS VP of FUD deserves a raise for this one... (Score:1)
1. Microsoft hired a known-biased "independent" lab to produce some obviously skewed results. If corporate America believes this, that's a bonus
2. The lab does tests on hardware for which there is not solid evidence that Linux is better.
3. The results are roundly drubbed by anyone with half a brain. The labs general conclusions are shown to be invalid.
4. The press examines the data and announces that Linux does not scale well and is not well-supported.
Microsoft did a masterful job of creating the perception of shortcomings in Linux.
Finally, a decent article from ZDNet (Score:2)
So, all you gurus out there, it seems there's a definite need for a "Highend-Server-Tuning-HOWTO". Any takers?
Also, I'd love to see the same test run on NT, a properly tuned Linux, Solaris 7 (which is supposed to be Much improved over 2.6 on x86 hardware), and Netware to see how many OSes beat NT, when Mindcraft claims the other way around. Unfortunately, I don't have the money for a quad-Xeon box or Netware, or I'd make my website useful for once
Mindcraft called RedHat tech support? (Score:1)
I have a great idea. Let's benchmark installing NT 4 against Linux. What we'll do is arrange, in advance, for some Linux gurus to show up to install Linux on our test box.
Then, we'll shoot a call in to Microsoft front-line tech support and ask them to send over an NT guru to install NT 4 for us.
I bet the MS guy will laugh about as hard as the RedHat tech support guy laughed when Mindcraft made the same request of him.
Then we'll publish results that the NT product wouldn't even jump out of the shrinkwrap on it's own, but the Linux installation went flawlessly in 30 minutes.
Heh heh, I really wish someone would do it, and publish it.
Commissioned by Microsoft...???? (Score:1)
So how much did Bill pay Mindcraft, Inc. to change the outcome in their favor?
Did they also mention that NT crashes faster than Linux? Funny - I have a friend that works as tech support at an ISP, they have 4 FreeBSD servers, 6 Linux servers, and about 20 NT servers. Each of the FreeBSD servers holds 300+ users and works like a charm, the Linux boxes hold about 200 users with no problems... NT? Holds about 50 before it starts to crawl... So the company has to have 4 times as many NT servers to accomodate those users who insist on using it. NT is not ready for primetime...
The Story has a point (Score:1)
tuning howto (Score:1)
Now that I think about it, this is the best way to document _all_ of linux. An organized site where anyone can write in a problem they've had, as well as a solution. As the number of users grows, more and more people can write in problems and fixes, until finally anyone looking for an answer can search and find an answer. Does something like this exist yet?
The cat's out of the bag... (Score:1)
ZD's PC Week [zdnet.com] says a similar configuration got them 150Mb/s - 160 Mb/s, depending on the client load. This is still not 217 Mb/s, but it does indicate a poor configuration of samba in mindcraft's test.
Or "creative" video editing :) (Score:1)
OTOH, I'm believing more and more that MS has perfected the use of deniable "mistakes" that just happen to make big initial splashes in the the press, but are quietly found to be wrong later.
"Expert Witness" (Score:1)
Amazingly, the other side also has an expert witness with great credentials who (surprise!) disagrees with everything your witness says.
Quad SMP for newbies (Score:1)
running a quad Xenon system with 4GB of memory. Complaining that a newbie can't find the tuning info for it is gratuitous.
Quad SMP for newbies (Score:1)
I have been tweaking MS-DOS/Win3/Win95 for over
a decade, over 5 years on this box. I have been
tweaking Linux for less than 6 months on the same
hardware. Get a clue, Mindcraft.
I think its a challenge (Score:1)
Keep in mind that the server used was so high-end that most Linux users would consider it a waste of money. The point is, Linux doesn't *need* high-end equipment to get high performance. NT does. In fact, you could probably run the same test with 3 of the processors removed and only 64 MB of memory and the Linux figures would stay the same while NT would be crippled. Thats my opinion at least.
-- UOZaphod
Tuning Linux, really? (Score:1)
Tuning is about optimising your machine/code for the conditions that it's running in. Nobody runs it under similar conditions, therefore everyone needs different tweeks. It depends upon your load, the types of request, how much memory you have, your bandwidth, back end CGI processes, etc, bleeding etc. And don't tell me that Linux is running fast enough as it is, next you'll be telling me that nobody would ever need more than 1GB of memory.
The website that I'm partly responsible for at my (very big) company could always be faster. Such is life.
Cian
The cat's out of the bag... (Score:1)
"Linux support lacking" (Score:2)
That said -- I do think that Mindcraft's attempt to gain support was a token attempt only.
If we want to keep the reputation as being a better source of support (via usenet) than Microsoft, then our community is at fault for failing to answer or to ask for clarification.
It does not matter that the invididual was a private individual. It does not matter if it was for a private box, commercial box, or for a publication. From what I heard no answer or request for clarification was ever made via usenet. If we're only going to help article writers then we're a lousy source of support.
Don't justify why we failed, accept the hit and produce a solution. I've thought of the knowledgebase solution but I never thought to mention it and see if anyone else is interested. My bad. I would like to see a knowledgebase implemented and I'd like to contribute to it.
HW vendor support is extreme bias (Score:1)
Put it this way: Given 2 reasonable people and 2 disassembled bicycles in Maine. One person is given the build instructions, the other isn't. I'll let you decide who will get their bike constructed first and ride it all the way to San Jose (or whatever).
A rebuttal by Alan Cox... (Score:1)
Kudos for Ziff-Davis (Score:1)
He is right ya know... (Score:1)
But it states NT wins... (Score:1)
well written (Score:1)
ZDNet has a point (Score:1)
I think the most VALID test bed design would be do this whole test again, but with OUT OF THE BOX configurations of both systems. This would set the constant to compare by. Then have both system's *tweaked* to the max and test them again.
This would do two things, one it would not create falsified results due to an uneven playing field, and it would also let IT management see what can be gained by hiring talented Linux admins and TRAINING their own admins. The more they know, the better off the company's servers will run!
Amazing pro-Linux from ZD! (Score:2)
I must say that was a very well written article. It pointed out the M$ sponsoring of the "benchmark," which is a key thing to note about the testing: M$ paid for it all.
The fact that they showed how it is possible to find the information that Mindcraft claimed it was not able to get proves that Mindcraft really didn't try all that hard to do the optimizations.
I'm glad that ZD is exposing M$ and it's lackey companies like Mindcraft for who they really are!
The Story has a point (Score:2)
RB
So I need to spend HOW much for the box? (Score:1)
From the Dell site the config they used comes to $40,617, except Dell only supply them with EDO RAM, so you have to add on the cost diff between 4GB EDO vs 4GB 100MHz ECC SDRAM.
No. Please let the experts handle this. (Score:1)
> I/O! Otherwise all I/O would stop while one was > in progress.
Um, think you missed that letter "A" in front of Synchronous - the guy said it *needs* to be ASYNCHRONOUS not synchronous. HTH.
-- laural
I think its a challenge (Score:1)
"What is the best performance which can be gotten
for a certain amount of money?" or even "What is
the least amount of money which can be spent to
get a certain level of performance?"
How about issues of reliability and expansion,
for this clustering makes more sense. In that
a single PSU, m'board, memory will not take down
the whole system. (Indeed the way NT was set up,
so as to have each NIC assigned to a specific
CPU, is part way to being a cluster anyway.)
Maybe someone should do some tests with Compaq
harware, after all they bought DEC and DEC had
their VAX machines clustering at least 13 years
ago.
Hold on, though, didn't some people from DEC
work on NT...
But it states NT wins... (Score:1)
NT won because RedHat screwed up by not
recognizing a need to support Mindcraft's
efforts and NT won because Linux still has
a beta RAID driver. The former can be solved
right away;
I suspect that the way Mindcraft approached
asking Red Hat was not disimilar to the way
they approached the newsgroups for support
with Apache.
Note that they did get a followup asking for
more information (and the output of netstat).
Did they follow this up? either they didn't or
dejanews has lost the posts...