Red Hat/Apache Slower Than Windows Server 2003? 628
phantomfive writes "In a recent test by a company called Veritest, Windows 2003 web server performs up to 300% higher throughput than Red Hat Linux running with Apache. Veritest used webbench to do there testing. Since the test was commisioned by Microsoft, is this just more FUD from a company with a long history? Or are the results valid this time? The study can be found here."
Just like the samba benchmark (Score:5, Informative)
Easy (Score:5, Informative)
*ahem* (Score:2, Informative)
Ahem... from the Article (Score:5, Informative)
At least they're up-front about it these days.
Other Veritest-Microsoft fun:
http://www.veritest.com/clients/reports/microsoft
http://www.microsoft.com/windowsserversystem/fact
http://www.gotdotnet.com/team/compare/veritest.as
In short, this is a company paid by Microsoft to make reports/whitepapers that make Microsoft look good. Nothing wrong with that as long as everyone's aware
Fair testing... (Score:2, Informative)
Re:Just like the samba benchmark (Score:5, Informative)
This report was written in April 2003, according to the first page. They used the most recent version of RedHat available to them.
This report may be two years out of date, but I can't see any signs of bias in its production.
Re:Just like the samba benchmark (Score:5, Informative)
I remember installing CentOS-3, based on RHEL3, on a server and having terribly slow disk performance with my raid adaptor. Running "yum update" to get the current patches yielded about a 10x speedup. Yet the Windows server gets a dozen or so undocumented registry tweaks.
In the SSL comparison, they're using the fastest (though slightly less secure) choice of encryption algorithms in IIS and the slowest in Apache. They're comparing RC4+MD5 to 3DES+SHA1.
And they decided to include ISAPI in the benchmarks without including the apache equivalent. All they test in apache is CGI. So again it's IIS's fastest option versus Apache's slowest option.
This is new? (Score:5, Informative)
Re:Easy (Score:5, Informative)
However, this might be more an effect of the underlying operating system than the actual server program. I haven't seen a comparison of Win32 Apache versus IIS, so I don't know.
Re:IIS is always faster. (Score:5, Informative)
You're shooting for a Funny mod, right? The biggest "advancement" in IIS 6 is that instead of IIS 5.X that that ran 100% in user-mode, IIS 6.X runs as a kernel module [certcities.com]
Which is a cute trick for gaining performance at the expense of security (kinda like the various Linux kernel-web-servers like khttpd)."But why would you believe that? I mean it's not like it's easy to find out.."
Indeed you are correct that it's not easy to find out. Leading security sites all report that it is NOT more secure as you allege. For example, the current rating of IIS 6report from Secunia, (one of the top couple security companies [slashdot.org] as opposed to merely your anecdotal rumor:
In contrast, Apache 2.X has the much better rating: "Apache 2.0.x with all vendor patches installed and all vendor workarounds applied, is currently affected by one or more Secunia advisories rated Less critical"Re:Let's be reasonable (Score:4, Informative)
Apache was never optimized for serving lots of small, static files so I can easily believe it falling behind in some benchmarks, but not 300%.
It doesn't take much computer to saturate a lot of bandwidth, which is why most people don't care, but big sites will often have a Zeus (or similar) server set up for serving images precisely because Apache isn't as good for that. But you've got to be huge before you get to that point.
Dynamic content put Apache where it is. It has the support, the tools, the libraries, and the widespread expertise to do dynamic content pretty damn well. It's not better than everyone at everything there either, but it's a very good solution for most cases.
Re:Just like the samba benchmark (Score:1, Informative)
I still see lots of biases, many of which can't be explained away as being the result of ignorance, laziness, or just knowing all the undocumented ways to tune windows.
Re:Just like the samba benchmark (Score:0, Informative)
Re:Just like the samba benchmark (Score:5, Informative)
1) The algorithms used in SSL are listed on page 33 of the pdf linked to. Both linux setups use 3DES+SHA1 and windows uses RC4+MD5 (as parent said).
2) This [hn.edu.cn] page (found via google) has a table comparing ciphers about 2/3 of the way down. RC4 appears to be about 2-3 times faster than 3DES.
3) This [ottawa.on.ca] email contains a comparison between MD5 and SHA1. MD5 appears to be 2.5 - 5 times faster than SHA1.
Re:"...the test was commisioned by Microsoft" (Score:3, Informative)
Trolls used to put long strings in which would stretch the page way over.
Learn how to use HTML links; like ImmortalFumbles [pandora.be] that, which you code like
<a href="http://users.pandora.be/vdmoortel/dirk/Physi cs/ImmortalFumbles.html">ImmortalFumbles</a>
(Slashdot will iinsert spaces in this of course.)
Also, your original URL had a trailing / which made it bad.
Re:Just like the samba benchmark (Score:3, Informative)
Re:Just like the samba benchmark (Score:5, Informative)
Speaking as someone who has quite some experience in cryptographic algorithms, I back up parent and grand parent. The benchmark is completely biased in that Veritest really ends up comparing 3DES+SHA1 with RC4+MD5. This unacceptable, I invite slashdoters to complain to Veritest:
Veritest1001 Aviation Parkway, Suite 400
Morrisville, NC 27560
Tel 919-380-2800
Fax 919-380-2899
E-Mail: info@veritest.com
Re:May not be FUD (Score:5, Informative)
You are mistaken on some Apache concepts and how threads (?used to?) work on Linux.
This is because for each request, Windows must create a new process (the CGI program), and destroy the process when the request is complete. While the execution time is low, the process management overhead dwarfs the actual page runtime, because Windows doesn't do that sort of thing quickly. This is why CGI has long been blacklistedon Windows systems by good web devs, and this is one reason that Apache 1.x was such a dog on Windows. Apache 1.x creates a new Apache process for each request.
No.
Now Linux, on the other hand, creates processes about as fast as it creates threads, which is to say, really damn fast.
Yes, but only because pthreads does this by creating a new process (that just happens to share some things with its parents, like address space). Ergo, creating threads is just as fast as creating processes because they are nearly the same thing.
The NPTL in 2.6 might have changed this, but I have not read the docs yet.
Yet Apache is still back here creating a process or thread for each and every request (note that there are some ways to speed things up. FastCGI comes to mind, but I don't want to get into the gory details that I don't know enough about). This is not the brightest way to do it in terms of performance, but then, Apache appears to have been designed for universality and configurability over raw throughput.
No, Apache does not create a new process for each request. It creates a pool of child processes which sit waiting for requests. The parent monitors this pool and creates new spare children when too many child processes are busy. This way, most of the time a request comes in there is already a child process sitting idle waiting for work.
CGI does indeed require forking a new process, but there are already great ways to handle this. mod_perl, mod_php, mod_python all do it by embeding the interpreter inside the server. FastCGI keeps a version of the program running (much like apache does with its spares).
You are correct in that your description isn't the brightest way to do things. That's why operating system designers solved these problems years ago.
For static content, again, Apache creates a new process or thread for every request (with some exceptions). If you'll forgive a bit of an oversimplification, it's like writing a program that prints text to the screen. One program calls printf() in a loop. The other program executes a second program which itself displays just one line, and runs that in a loop.
Again, no. Apache will usually not need to create a new process or thread for every connection. The correct analogy would be the other program spawning the required number of children, and then asking them to all printf at the same time.
Re:Let's be reasonable (Score:4, Informative)
It could be. However, this test is severely flawed in that they performed registry level optimisations to the Windows setup, yet equivalent optimisations that are well documented for Linux were not performed. Therefore, we don't know.
Re:Just like the samba benchmark (Score:5, Informative)
Strange, they have a press release [lionbridge.com] on their website dated April 6, 2005 about the report being commissioned by Microsoft. Either Microsoft got ripped off by recycling an old report, or one of those dates is wrong.
Re:Just like the samba benchmark (Score:3, Informative)
Different report. That press release talks about Windows Server 2003 vs. RHEL 3.0 -- Microsoft must have asked them to produce a newer version of the report
Re:How to tell if you are a linux fanatic. (Score:3, Informative)
I have a choice of Larson, SDI, Zeh and Easycopy as linux vendors to print to the 42 inch non-postscript printers in my workplace - very much a niche market but still covered.
Microsoft never entirely took over the workstation market, and linux boxes have been used as cheap unix workstations for years.
Visual memory vs other kinds - some people find mousing through a lot of menus or the registry easier than flat config files, while I'm the other kind - valid point taken.
I run commercial software on 24 dual Xeon linux machines that costs almost as much each year as it did to buy the machines, but it is used to do things which make money for the company. It runs on solaris and AIX machines in the place as well. If I have a problem with it in the middle of the night there are people I can call to solve the problem - but normally emails with a one day lag due to time zones are good enough. As far as the company that sells the software is concerned we are a small operation - there are people with very big clusters out there.
Even from a disk image it take more than "a couple of minutes" to set up a windows server, even on something small like NT4.
It's unix - I know this!
Third party software is everywhere, and it gave MS Windows the ability to get onto the net. Why re-invent the wheel when you already have something decent in the same group?
X is old news and VNC has been around for a few years too - in a wide variety of different situations both appear to still be a better solution than windows terminal services.
Yes, but I'm doing it from work! However, Im doing it on a Saturday night while rebuilding a disk array - must be a masochist as specified above :(
Good point - all it took was one idiot giving all mail users shell access, turning on telnet and another idiot using "coffee" as a password and I had to rebuild a hacked box. You can set up an insecure system with just about any OS if you don't have a clue - there are plenty of people who use linux who don't have a clue, we all have to start somewhere - the learning curve is there, so if you don't know what to do you have to follow the docs or find out.
Have you ever seen other databases? MS Access vs most other databases is a similar comparison to MS Notepad vs most word processing software. Similarly you can still do decent work in MS Notepad, and sometimes that's all you need. MS Access doesn't even have a stable scripting language - I've learned two seperate scrip
Re:Stop whining and help speed up Apache! (Score:1, Informative)
Re:Just like the samba benchmark (Score:5, Informative)
openssl speed rc4 md5 des-ede3 sha1
(Get OpenSSL here [shininglightpro.com] if you are using Windows). You will see that the first two algorithms are much faster, especially for larger blocks.
I say this shootout is rigged.
Re:Just like the samba benchmark (Score:1, Informative)
~ $ openssl speed rc4 md5 des-ede3 sha1
Doing md5 for 3s on 16 size blocks: 3229671 md5's in 2.98s
Doing md5 for 3s on 64 size blocks: 2721010 md5's in 2.99s
Doing md5 for 3s on 256 size blocks: 1858659 md5's in 2.99s
Doing md5 for 3s on 1024 size blocks: 816811 md5's in 2.99s
Doing md5 for 3s on 8192 size blocks: 131011 md5's in 2.99s
Doing sha1 for 3s on 16 size blocks: 3225174 sha1's in 2.99s
Doing sha1 for 3s on 64 size blocks: 2158082 sha1's in 2.98s
Doing sha1 for 3s on 256 size blocks: 1167085 sha1's in 2.98s
Doing sha1 for 3s on 1024 size blocks: 438103 sha1's in 2.99s
Doing sha1 for 3s on 8192 size blocks: 63982 sha1's in 2.99s
Doing rc4 for 3s on 16 size blocks: 46931949 rc4's in 2.99s
Doing rc4 for 3s on 64 size blocks: 13189054 rc4's in 2.99s
Doing rc4 for 3s on 256 size blocks: 3364646 rc4's in 2.98s
Doing rc4 for 3s on 1024 size blocks: 861101 rc4's in 2.99s
Doing rc4 for 3s on 8192 size blocks: 108304 rc4's in 2.99s
Doing des ede3 for 3s on 16 size blocks: 3450802 des ede3's in 2.99s
Doing des ede3 for 3s on 64 size blocks: 864132 des ede3's in 2.99s
Doing des ede3 for 3s on 256 size blocks: 215751 des ede3's in 2.99s
Doing des ede3 for 3s on 1024 size blocks: 53903 des ede3's in 2.99s
Doing des ede3 for 3s on 8192 size blocks: 6752 des ede3's in 2.99s
OpenSSL 0.9.7e 25 Oct 2004
built on: Sun Dec 19 09:43:29 UTC 2004
options:bn(64,64) md2(int) rc4(ptr,char) des(idx,cisc,16,int) aes(partial) blowfish(ptr2)
compiler: gcc -fPIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DOPENSSL_NO_KRB5 -DOPENSSL_NO_IDEA -DOPENSSL_NO_MDC2 -DOPENSSL_NO_RC5 -m64 -DL_ENDIAN -DTERMIO -O3 -Wall -DMD32_REG_T=int
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
md5 17340.52k 58242.35k 159136.02k 279737.28k 358943.85k
sha1 17258.46k 46348.07k 100259.65k 150039.29k 175297.84k
rc4 251140.86k 282307.51k 289043.41k 294905.49k 296731.23k
des ede3 18465.83k 18496.47k 18472.33k 18460.43k 18499.13k
Re:IIS is always faster. (Score:1, Informative)
Right now, it looks as if IIS *6* has a way better security track record than Apache.
Re:Accelerating Apache (Score:2, Informative)
Reading you link...
So I presume... nolike noatime (Score:4, Informative)
noatime disables the update of the "last accesS" field of files, and improves the performance a lot for some workloads. If you check the latest article about the kernel.org servers, they found that they reduced the system load to the half by just using this option
This analisys is biased. Who cares, anyway?
Re:Just like the samba benchmark (Score:5, Informative)
Surprisingly (controversially?) enough, some EULAs forbid public criticism - I wonder if such clauses would ever be found valid in court, I seriously hope not - judges should declare void in whole any EULA that includes any anticonstitutional demands.
Now that I think about it, I seem to remember that M$ used to include a non-comparison clause in many of its products' EULAs, this "licensed comparison" tells me it probably still does.
Re:Just like the samba benchmark (Score:2, Informative)
http://www.veritest.com/clients/reports/microsoft
Re:Just like the samba benchmark (Score:3, Informative)
Anyway here is the trick. First off IIS used ASP which does not use a mod interface like cgi scripts which means the engine can run in IIS itself. This makes IIS very fast.
We have the same thing as ASP in the FOSS world called PHP. Zend is fast because the engine also runs in the same space as apache.
Or I have seen in older 1999 benchmarks that MS will just use a static html to show how fast there platform is and ignore dynamic content. Also MS used a patch that was not made public which bound the I/O of 4 nic card to 4 cpu's in order to make I/O overhead very low. I think it may be standard in Windows2003 but its dumb tricks like these no one uses in a production environment anyway
But I do not trust any benchmark that has a "Migrate from Unix to Windows" link on the right side of the report. Nooo its not biased
However someone can point to a benchmark by the Linux kernel team and taking it as a grain of truth that its faster then net or freebsd. Its a dual standard.
Re:IIS is always faster. (Score:3, Informative)
See so theres more to securing your box than turning off one tool, you have to know how to look up the issues which you can do easly on Apache's site right here: http://httpd.apache.org/bug_report.html [apache.org] and its linked right off the front pages of the web servers site.
Then theres Microsoft's site for iis who's security link, links to this wonderful page http://www.microsoft.com/security/guidance/prodte
Yup that gives you a warm and fuzzy feeling all over!
Figures (Score:3, Informative)
They shut off access logging in IIS. As far as I could see, they left logging on for Red Hat. This means that lots of disk writes were being generated on Linux but not on Windows. As http request volume goes up in their tests, the RAID write-cache could eventually fill up (only under Linux), at which point the webserver starts blocking while waiting for disk I/O to complete.
Figures that right after submitting this I see that they turned off access logging in Apache. Doh!
Re:Just like the samba benchmark (Score:3, Informative)
I found another flaw on that same page.
VeriTest also write that Windows 2003 was using RSA key exchange and Red Hat was using Diffie-Hellman (DH).
But DH [wikipedia.org] is vulnerable to a Man-in-the-Middle attack so SSL uses RSA to perform the authentication.
So Red Hat is doing RSA and DH, whereas Windows is doing only RSA!
Using OpenSSL's ssltest program I noticed that DH+RSA was 50% slower than RSA:
$ time ./ssltest -num 1000 -tls1 -cert server.pem -key server.key -c_cert client.pem -c_key client.key -cipher "RC4-MD5:@STRENGTH" -client_auth -server_auth -CAfile cacert.pem
./ssltest -num 1000 -tls1 -cert server.pem -key server.key -c_cert client.pem -c_key client.key -cipher "EDH-RSA-DES-CBC3-SHA:@STRENGTH" -client_auth -server_auth -CAfile cacert.pem
$ time
And I would not be surprised if Windows 2003 was using SSLv2 (faster and insecure) while Linux was using TLS1! Because that is another parameter that VeriTest is not disclosing.