

HP Publishs First Linux TPC-C Benchmarks 208
The first ever official TPC-C benchmark on a Linux system has been published.
This was run on a cluster of 32 HP servers with Intel Xeon CPUs, running Redhat Linux and Oracle RDBMS. The system had over 18 terabytes of storage, and cost over 2 million US dollars. Performance was higher than a similar system running on MS Windows.
Just imagine... (Score:1, Redundant)
Oh never mind.
Re:Just imagine... (Score:1)
TPC-C (Score:1, Funny)
Re:TPC-C (Score:1)
Re:TPC-C (Score:1)
Re:TPC-C (Score:1)
I wonder how a linux system with some free database software would rank on here...
Not really. (Score:2)
ANd if you look at it on a cost basis, it's a HUGE contender.
Windows or Oracle (Score:3, Interesting)
Re:Windows or Oracle (Score:1)
Still not the fastest price/performance.... (Score:3, Interesting)
http://www.tpc.org/tpcc/results/tpcc_price_perf_r
It's interesting, though, that Red Hat had the cheapest Oracle implementation.
Re:Still not the fastest price/performance.... (Score:2, Interesting)
It takes only a small leap of faith to believe that whatver HP did to their old #5 system to push it down into slot 6 could also be applied to their #s 1, 2 and 3 systems.
For example, and _purely hypothetically_, assuming a 7.26% improvement for those systems too on migrating to Lunix would give results of
$12.14, $13.09, $13.94
Which would leave the table at
Lunix $12.14
Doze $13.02
Lunix $13.09
Lunix $13.94
Doze $14.04
Doze $14.96
The majority of the rating improvement (>6%) was through diminished _cost_, which is pretty much guaranteed by migration to Lunix. Only a small part (1%) was through improved performance.
So the leap of faith really isn't that huge.
Maybe HP were hoping to convert their #5 Doze machine into a #4 machine, ousting IBM, but failed.They currently don't actually need to improve their #1 position.
Only time will tell.
THL.
"Publishs" (Score:3, Funny)
This just in: ditors at Slashdot ar no longr prmittd to us th lttr '' in articl titls!
Re:"Publishs" (Score:1, Offtopic)
Correction... (Score:2)
They did spell Benchmarks correctly at least.
Inconclusive (Score:4, Insightful)
On the other hand, the difference in performance is 17.21 vs. 18.46, i.e. approximately 6%. I think that the result is inconclusive (except that Linux can complete in this area at all).
Your point? (Score:1)
The nice thing about statistics, benchmarks, or anything else that tries to assign a number with some sort of value to something that's much more complicated is there are so many factors in the complicated thing, and thus ways to fsck up. Therefore, there are equally many ways to fudge.
Re:Inconclusive (Score:2, Informative)
I noticed a few things when looking at the break-down of costs:
... and a whopping 15" CRT. The clients only make up 5% of the overall cost though.
For the clients, a 1G 133Mhz SDRAM DIMM is $880! ouch. The cost per client is $7172, for which you get a dual 1.4Ghz Pentium III with 4Gb of RAM
The Oracle software is $35000/CPU, for 3 years. What happens after 3 years? Does Oracle lease the software or something? Oracle support is shown as 2,000 * 24. Where does the 24 come from?
Re:Inconclusive (Score:1)
"the difference in performance is 17.21 vs. 18.46, i.e. approximately 6%"
!?!?
Those weren't the measure of _performance_.
If you'd read the summaries you'd know that the majority of the increased rating was due to a difference in _price_, and only marginally due to an increase in _performance_.
Yes, I'm a pedant. Sue me.
THL.
Re:Inconclusive (Score:3, Insightful)
That's exactly what Linux needs: Marketing.
Re:Inconclusive (Score:2)
Re:Inconclusive (Score:2)
* Different transaction monitors
* Different Oracle optimizations
* Newer/faster/cheaper components in the machines
Still interesting that HP is touting Linux, but the point is accurate that the difference is inconclusive. I find it also interesting that the putatively cheapest solution (the only difference here would be attributed to OS price) did not make the top 10 price/perf ratio.
more TPC-C scores... (Score:2, Informative)
Comparable cost between windows and linux cluster (Score:5, Informative)
Linux cluster:
Windows cluster:
Note that the number of clients in the windows tests is higher 24 instead of 16), with smaller CPU's. Also, the server's aren't identical.
Besides from the small differences in setup, it's plain that hardware-costs greatly outnumber software costs. Yeah, linux has a small bit more performance (less than 1%) for a bit lower price (6%) but these aren't real shocking numbers. Of course, I'll get flamed for not bashing microsoft, but the difference really isn't that big.
Re:Comparable cost between windows and linux clust (Score:3, Interesting)
Don't be silly.
From this FAQ [tpc.org] you'll see:
In general, TPC benchmarks are system-wide benchmarks, encompassing almost all cost dimensions of an entire system environment the user might purchase, including terminals, communications equipment, software (transaction monitors and database software), computer system or host, backup storage, and three years maintenance cost. Therefore, if the total system cost is $859,100 and the throughput is 1562 tpmC, the price/performance is derived by taking the price of the entire system ($859,100) divided by the performance (1562 tpmC), which equals $550 per tpmC.
Most people would focus on the hardware cost, but in reality the highlighted maintenance cost took the precedence.
Most midrange UNIX server has outragous maintenance cost. The maintenance cost of a UNIX server in the third year could be exceeding the cost of the hardware itself. It's due to the fact that older parts are difficult to find, thus make maintaining older servers more difficult. Besides, they really want to cut older production lines in favor of newer servers production.
x86 platform is known to have flat and lower maintenance cost, due to the low cost hardware and high compability with older hardware, i.e. older parts can be found easily. That's why Microsoft could easily beat the TPC pissing races.
Doomsday finally comes to Microsoft when Linux is entering the database market. Although at this moment big corps are still offering Linux maintenance with cost comparable to UNIX package, that's not surprising when Linux engineers are not as abandon as MCSE. However, it'll not be the case in the future. I think Microsoft would eventaully lose this pissing race in the long run.
Re:Comparable cost between windows and linux clust (Score:1)
http://www.tpc.org/tpcc/faq.asp :
http://www.tpc.org/tpcc/detail.asp :
THL.
Re:Comparable cost between windows and linux clust (Score:1)
More maintenance years would only cause the result of $/tpc-c more favour to x86 platform, no wonder why MSSQL servers almost dominate top ten of this rating.
Thanks for the info.
Re:Comparable cost between windows and linux clust (Score:2)
it's not about not paying for the software (Score:5, Insightful)
The reason to use Linux is all aspects of its openness and compatibility with other systems. With Linux, you aren't locked into a single vendor. You use tools and APIs that have been around for nearly two decades and are available, in multiple implementations, from dozens of vendors. And you control how you upgrade, when you upgrade, and what path you follow with the software. And if you don't like Linux anymore, you can switch to any of a dozen other, compatible platforms.
With Windows, you are locked into a single, proprietary implementation and Microsoft has you by the proverbial precious body parts; there is no other vendor you can get a compatible implementation of Windows or all the Windows libraries from. Every couple of years, Microsoft completely changes their computing paradigms to ape what they perceive is a threat from some other company, and when the threat is gone, they just drop the initiative and move on to the next thing.
You can get stability buy paying a premium to a company like IBM, which is committed to providing it, or through open systems available from multiple vendors or open source, where you control your future. But building a large, long-lived infrastructure on Microsoft platforms is a costly folly--the company has proven that they will change approach every couple of years and that they will force their customers to move along.
Re:it's not about not paying for the software (Score:2, Insightful)
This benchmark tested Oracle. If you put 18TB of data into an Oracle database you are locked into a single vendor anyway.
And if you don't like Linux anymore, you can switch to any of a dozen other, compatible platforms.
No, you can switch to whatever platforms Oracle supports. Because you have 18TB of data in it.
With Windows, you are locked into a single, proprietary implementation and Microsoft has you by the proverbial precious body parts; there is no other vendor you can get a compatible implementation of Windows or all the Windows libraries from.
I'm using an Oracle database. Why do I need compatible implementations of all the Windows libraries if I decide I want to migrate that database cluster to another host OS?
Re:it's not about not paying for the software (Score:2)
But being locked into 2 vendors is worse than being locked into one single vendor, don't you agree?
Also, migrating between databases is not *that* hard, it's all SQL after all.
Does PgSQL scale? (Score:2)
Also, migrating between databases is not *that* hard, it's all SQL after all.
But does PostgreSQL, Oracle's closest free competitor, scale to an 18 TB database under the loads they're testing?
Re:Does PgSQL scale? (Score:2)
So you are not locked into a single vendor. Period.
Re:Does PgSQL scale? (Score:2)
Re:Does PgSQL scale? (Score:2)
Re:Does PgSQL scale? (Score:2)
Re:it's not about not paying for the software (Score:2)
Another reason to use Linux is that had Linux finished behind Windows on this benchmark, it wouldn't take long before someone figured out and fixed the Linux kernel and released the patch. I don't remember the exact test, but this very situation happened a few years ago.
I can't imagine Microsoft releasing a patch to address this (admitedly small) performance deficit, and if they did, who knows what other damage it would cause. Again, I don't remember the exact situation, but Microsoft has definitely released service packs that were considered to be avoided.
Re:it's not about not paying for the software (Score:2)
Percentage-wise, you may say so, but I'll take the $152,549 difference anyday (=$4767.15 per CPU at 32 CPUs...)
With the current prices, that difference in money will buy you 200 brand new desktops for your office workers...
Re:it's not about not paying for the software (Score:2)
Every couple of years, Microsoft completely changes their computing paradigms to ape what they perceive is a threat from some other company, and when the threat is gone, they just drop the initiative and move on to the next thing.
Who the fuck mods this blather up as insightful?
I can still run 16 bit DOS programs on windows. I can still use DDE for IPC. COM and COM+ haven't gone away with
--
Obviously it is Allah's will that I throw the unix box out the window. I submit to the will of Allah.
Re:it's not about not paying for the software (Score:2)
Yup, you can still run all that stuff, and I didn't claim otherwise; Microsoft would be foolish to alienate their customers that blatantly. Your 16bit programs will run in all their 16bit glory.
What you are missing is that those APIs are dead or dying--programs using them can't take full advantage of the system, Microsoft doesn't want you to use them anymore, people usually don't develop for them anymore, and there are fewer and fewer tools. That's what "changing paradigms" means.
UNIX and Linux provide continuity and growth that goes beyond merely backwards-compatible support. You should find out about it some time.
Re:Comparable cost between windows and linux clust (Score:3, Insightful)
Huh? The cost of the Windows software adds $150K more. That is a lot of money. Think of it this way: if I have two laptops studded with diamonds that cost 2.3 mil each (before the OS is installed), and the OS for one is free while the OS for the other is 150K - the latter seems very expensive when you look at the software budget
I think that you should break out the costs of these systems and look at the hardware and software seperately. "The systems cost the same in hardware, but there is a $150k difference in software." is a much cleaner analysis.
Re:Comparable cost between windows and linux clust (Score:1)
A new system is certainly going to cost less than an older one. Not true of cars unfortunately, but true of computers.
What would be nice is a true apples-to-apples test on the same hardware.
Re:Comparable cost between windows and linux clust (Score:2)
Finally, it's not about the softare cost - it's about the bottom line: price/performance. These reports can't really be scientifically compared. One can speculate that Windows would have beat the price/performance ratio of the Linux system given an even playing field, but it's just that - speculation. The real point here is that Linux is _competing_ in the enterprise and has been taken seriously enough to actually get benchmarked.
Re:Comparable cost between windows and linux clust (Score:1)
Re:Comparable cost between windows and linux clust (Score:3, Insightful)
I'm not bashing you, I just don't see your point.
If you get a car for 20000$ at dealer A and the same car for 19500$ at dealer B, will you say "Hey, it's just a small difference" and buy from dealer B?
Let's not forget, this is about database-servers here and both run the very same database (Oracle). The underlying OS is irrelevant, you don't have to run MS Office or "the Sims" on this thing.
So please tell us stupid Microsoft-bashers what is your point.
Re:Comparable cost between windows and linux clust (Score:2)
And the difference between the two system costs from an OS licence standpoint is far less (see the full report). To correct your analogy:
$20000 dealer A
$19900 dealer B
It's not based on money the initial purchase. It's based on the TCO, how it will integrate with your existing infrastructure, and whethor or not you prefer one of the other (for a variety of reasons).
Not available for six months! (Score:1)
hmm... different setup, little difference (Score:1, Insightful)
How does it scale (Score:2, Insightful)
I do not hazard a guess, but it is surely interesting what Linux can do best, instead of comparing it to Windows, only.
Re:How does it scale (Score:2, Informative)
AL
A bit flawed.. (Score:4, Insightful)
oh, that's easy! (Score:2)
It happens every day! All you have to do to see it is take whatever M$ junk you are using and replace the software. All the PitA you see on a desktop with poor security and stability, obscure and changing protocals, propriatory data formats, unmodifiable applications, ad nauseum, translate.
We have at least one multi-processor Dell "server" where I work and the folks responsible for it are not happy. Keeping it up has been a quality of life killer. The problem is not in the hardware.
Re:oh, that's easy! (Score:1)
I already know that Linux is a great solution - I use it for every server I setup where I work.. I just mean that it would be nice to see some official figures of Windows getting a beating
Summary incorrect - 16 dual processor servers (Score:1)
sorry - summary was correct (Score:1)
I read the article, just not well
Off Topic? (Score:2, Interesting)
GNU/Linux impresses me a great deal. I've been running it for a few years now, mainly at home and in some positions when I can get away with it and its easy to take it for granted. But its articles like this that make me step back and think about what as really been achived by a bunch of mainly unpaid developers who provide all their source code for free.
Sure there a few places where Windows is more suitable but everyday, these few places get fewer.
So to any Linux developers and supporters who read this, thank you all.
HP follup on lkml (Score:4, Interesting)
Re:HP follup on lkml (Score:3, Interesting)
And it's a valid comparision? (Score:3, Insightful)
The Windows TPC benchmark was on last years hardware (IBM xSeries 370, released last year).
The Oracle TPC benchmark is on next years hardware (HP ProLiant DL580R - not available till May 2003).
Crowing about how performance is higher under Linux is FUD. It's not a fair comparison. Or didn't the story submitter understand that hardware always affects performance?
Re:And it's a valid comparision? (Score:1)
Yeah, TPC comparisions are always a bit of a game. And it takes some effort (and cost) to sort out the hardware and run the test.
That Linux has a point on the graph that isn't disimilar from other systems may help dispel some doubts about Linux scaleability and potential as a back office platform. IMHO that's more significant than whether today's TPC ratings put it slightly above or slightly below Windows.
Re:And it's a valid comparision? (Score:3, Informative)
The Oracle TPC benchmark is on next years hardware (HP ProLiant DL580R - not available till May 2003).
Err... are you talking about the TPC benchmarks refered to in the links? Both were run on 8 ProLiant DL580's with a total of 32 Xeon 900s. The Linux setup is marked in the full disclosure as being available March 5 2003, Hardware available now.
Re:And it's a valid comparision? - IBM?? (Score:3, Insightful)
Also, the clients are not only different in number, but the Linux clients are running a faster CPU as well (1400 Mhz vs. 1000 Mhz).
The only conclusion I think we can draw is that Linux plays "in the same ballpark" as Windows, performance-wise.
Re:And it's a valid comparision? - IBM?? (Score:2)
But if you read the specs, there were 2 CPUs per Linux client, and 48 CPUs per Windows client.
Run the benchmark non-clustered (Score:2, Insightful)
What I would love to see is Linux TPC-C benchmark running on a 8 way machine. Particularly the IBM x440. It scored 91,000 + tpmCs running Windows
http://www.tpc.org/tpcc/results/tpcc_result_det
Re:Run the benchmark non-clustered (Score:1)
This is not true if you use Oracle 9i and Real Application Cluster.
Price/Performance doesn't apply to many configs (Score:2, Interesting)
However, in reality, a lot companies have $10,000 or even $5,000 DB servers.
When you factor the cost of Windows into these systems the difference in Price/Performance is much greater than 6 percent.
Re:Price/Performance doesn't apply to many configs (Score:1)
Price/Performance was low (Score:3, Interesting)
Re:Price/Performance was low (Score:2, Informative)
The real challenge here is finding the money to build competing open source - based machines. MS has lots of monopoly money to dedicate to getting their products on the lists. Linux has no similar war chest.
Re:Price/Performance was low (Score:1, Interesting)
That PostgreSQL will scale (like Oracle).
OK. Take out Oracle, save a bundle. Put in PostgreSQL, now you cannot handle this transaction load. The result - Your price/performance is now worse.
I would be interested in seeing PostgreSQL run simultaneously on more than one node and access the same files (ie RAC) or comprimise a single database (ie distributed).
Re:Price/Performance was low (Score:2)
Upon Close Inspection .. (Score:4, Interesting)
Dangit. IBM, Dell, HP
Re:Upon Close Inspection .. (Score:1)
The two aren't equal for numerous reasons, like different kinds of hardware, varying level of redundancy, reliability of the hardware and ability to partition and hot-swap. DBA's that deal with hardcore realtime data and realtime backup don't use SQL Server. When you try to do things like realtime backup, with a clustered database setup, it gets very complicated. It's even harder to find DBA's with that level of experience and skill. That whole price/performance thing is a nice starting point, but it's just the beginning of the iceburg.
Re:Upon Close Inspection .. (Score:2)
Re:Upon Close Inspection .. (Score:2, Interesting)
- Microsoft
- AIX - 10 times
- HP UX - 8 times
- Compaq Tru64 UNIX - 4 times
- Solaris - 4
- OS/400 - 2
- Linux - 1
The rest are Windows because only now the Companies
use Linux for these benchmark.
It's a start. (Score:1)
Re: (Score:2)
Interesting, but I doubt it... (Score:1)
Besides, Oracle tends to appropriate, customize and rebrand, not recode. Examples: Oracle Application Server is simply Apache with a custom Oracle Module, JDeveloper is simply a cosmetically modified Borland JBuilder with custom Oracle EJB Components.
In fact, I suspect that Oracle is profoundly lazy when it comes to new development. Oracle Forms 6i is just a Java implementation of the Oracle Forms Runtime Client, everything else is the same, (but slower), and if you've ever seen the underlying PL/SQL in some of their industry targeted applications then you know they are all about code monkeys and are in no way concerned with elegance or even efficiency.
Dont make me... (Score:1)
Don't make me break out the "it's the hardware stupid" baseball bat and smack you upside the head....
Yes, Linux is cheaper, but the OS in these tests matters about as much as what the admin was drinking during the test...
drinking... (Score:1)
the combination of what and how much the admin was drinking can matter quite a bit !
How about PostgreSQL? (Score:2)
On a side note, doesn't anyone else find it ridiculous that you are not allowed to publish "unauthorized" benchmarks for the proprietary databases?
Not so Close in Cost. (Score:2)
When Windows XP2(?) arrives you will have to retrain your key people. Do test deployments, do a deployment plan, find replacement applications for those that are no longer working, work with and pay your inhouse developers to check, and redo your older homegrown programs, develop a hardware compatibility list with your vendors, repeat every three years.
Re:Not so Close in Cost. (Score:1)
Windows clients running W2K *Server* (Score:1)
I understand some of the subtleties of the differences between W2K Server vs W2K pro, but given a dedicated client app, I'd think the slight memory management issues wouldn't have that big of an effect on the performance.
Client Software
---------------
BEA Tuxedo 6.5 Teir 1
Microsoft Windows 2000 Server
Client Software
---------------
BEA Tuxedo 8.0 Teir 1
Red Hat Linux Personal
Almost similiar (Score:1)
I don't understand (Score:2)
20TB data and a 12GB tapedrive? (Score:2)
Disk Drives 976 18.2-GB 15K
Disk Drives 16 18.2-GB 10K
Total Storage 18054 GB
Tape Drives 1 12/24-Gigabyte DAT
Huh? How many tapes for a full backup?
And uh, 976*18.2+16*18.2=18054.4.
No RAID either?
How do transaction processing people protect their data?
At all?
Re:"Publishs"? (Score:1)
Jinx. [slashdot.org]
Re:THEY CHEAT! (Score:1)
(pretty sure its a typo.. but thats what it says)
Re:RedHat? (Score:1)
Nope, I agree. The power of open source solutions performance-wise is not that it's faster out of the box, it's that you have room to do all sorts of cool tweaking.
I like Redhat, but in my past experience, redhat isn't the best alternative if you want to do some serious tweaking.You can recompile most stuff to optimize it, find the best FS for your needs, tweak caching of stuff, tweak DB configuration, etc...
There is often more performance to be gained from this than you'd think.
Re:RedHat? (Score:2)
That said, on enterprise critical systems running Oracle the less local tweaking the better. The system is certified to run Oracle and there'd better be no local tweaks if anything go wrong ("I tweaked this IO-driver a bit... performance went up 20% when I skipped this sync bit").
Re:RedHat? (Score:1)
What I have to back this up is experience. I have seen alot of things, including (but not limited to) databases perform alot better after extensive tweaking, and recompilation of things like common libraries.
And although this might be fully possible to achieve with Redhat, I wouldn't exactly say that Redhat is very optimized out-of-the-box. So it would require alot more work to do the things _I_ want to do.
And btw - what's wrong with recompiling 'ls' with the '-march=i686 -mcpu=i686 -O3 -ffast-math -fstrength-reduce -fthread-jumps -fschedule-insns2 -fexpensive-optimizations -fomit-frame-pointer' flags? =P
Massive ignorance Re:RedHat? (Score:1, Informative)
I have done an extensive study on the effect of compiler flags on performance and I can tell you that to begin with some of them have near zero effect, some others interact badly (ie they speed things up when used separately but slow them down when used together), that some of the flags you give are already enabled when you use -O2 (BTW the -Ox flags are a just convenient way to enable a miriad of "minor falgs"). That -O3 is a double edged sword since in gcc 2.x it is equivalent: to "-O2 -finline-functions" thus making the code faster on small programs (sinced there is no the overhead of function call) but bigger (and thus it could slow large programs if it makes you run of TLB entries, cache or RAM), and it also slows things down when the program (eg the Linux kernel) has already had its significant functions inlined by the programmer (if you use -O3 on it the compiler will inline functions who are small but that the programmer didn't inline because they are rarely called, so you get no benefit but the enlarged size could make that two functions who were in the same page are no longer and thus you get TLB faults). Finally due to bugs on all versions of gcc's optimizer there are significant cases where -march=i686 will be significantly slower on the 686 family (PPro, PII, PIII, Celeron) than using plain -march=i386. Ah, it is not a foregeone conclusion that the Athlon behaves better with a program compiled for the 686 than for the 386: due to differences in architecture and timetable all the 686 family runs more than 10% faster on programs compiled for the 386 than on programs compiled for the Pentium.
Now that we are at it: RedHat 7.x are compiled with -mcpu=i686 (that means: optimize for the 686 family but using only 386 instruction so it runs everywhere). Using -march=i686 like you do only gives a 2% performance increase. For programs who have parts written in assembler (kernel, glibc, ssl) RedHat provides versions for the 686 and the Athlon and these are compiled with -march=i686 and -march=k7 respectively (ie full optimization).
Nearly forgot: the effect of the minor flags not included in -02 (except for -finline-functions who is part of -O3) is quite minimal when comparing the best combo of them to plain -O2 or -O3. From a memory it was couple percent at most. defintely not worth the risk since exotic combos are not so well tested and you have a chance of gcc generating wrong code.
Re:Massive ignorance Re:RedHat? (Score:1)
Re:RedHat? (Score:1)
Life is never easy when you're posting on /. ;)
Re:RedHat? (Score:4, Informative)
The Link in the article is Bullshit (Score:3, Informative)
That article link is like something a politician would say. Do you realize that Oracle is optimized for UNIX? It never should have been ported to Windows. Such a comparision is akin to saying "Apache 1.x runs better on Linux than on Win32!" No shit, Sherlock. Anyone who looks at the top ten results will clearly see that the point this article is trying to make means jack as far as the real numbers go.
By the way, I hear the latest Linux kernel is better at running Linux programs than CygWin!
Re:Might not be a good idea to be getting behind t (Score:1, Funny)
>Windows is not interesting to me. Its not penetrating the back end, even though Microsoft marketing pretends it can.
heh heh. you said "penetrating the back end".
Re:Might not be a good idea to be getting behind t (Score:1)
Whether or not that was a typo for shift it made me laugh.
Feeding the troll... (Score:2, Funny)
FreeBSD, specially compiled? (Score:2)
Sorry, BSD and Stephen King are dead. Haven't you been reading Slashdot?
Oh, and that Elvis guy. He is really dead.
A better question: How fast would it run on a FreeBSD compiled just to do that particular job?