OSDL Releases TPC Benchmark Tests For Linux 96
trialsboy writes "A news.com article announces that ODSL has released a set of workloads and test frameworks which conform to the TPC benchmark specifications." (Slashdot parent company VA Software is one of the OSDL sponsors.)
ODSL? (Score:3, Funny)
Re:But Around Here... (Score:5, Funny)
Re:But Around Here... (Score:1, Offtopic)
Don't you love the VA Software disclaimers? (Score:1, Funny)
Re:Don't you love the VA Software disclaimers? (Score:4, Funny)
You're new here aren't you.
benchmark handbook (Score:5, Informative)
Suhit
Question (Score:1, Interesting)
Re:Question (Score:1)
Re:Question (Score:3, Informative)
I think you are confused, this is a test suite, period.
Re:Question (Score:2)
Now, there is no excuse to say you don't trust the results because the suite of tools are readily available for you to run your own tests. It's as simple as that.
Does this mean you should trust any result quoting OSDL TCP results? Of course not. Does that mean you might be able to validate them for a change? Absolutely!
Ignorant question: (Score:4, Interesting)
Re:Ignorant question: (Score:3, Funny)
Re:Ignorant question: (Score:2)
Re:Ignorant question: (Score:4, Funny)
Re:Ignorant question: (Score:3, Funny)
Re:Ignorant question: (Score:1)
Yeah, Linux is great... (Score:5, Funny)
I don't know what I'm going to do now. When I tell poeple my server run Linux they don't look at me like some granola head. I'm actually starting to see Linux showing up on Intern's resume along side Microsoft Word and Excel as computer experience.
It getting to be cool, and I've never been cool before. I backed apple back in '94. I had a PC when everyone had a C64. Could it be, that for a shining moment in time, I could actually be doing something that the rest of the world considers cool?
Nah, back in your box Skippy.
Re:Yeah, Linux is great... (Score:1, Offtopic)
Re:Yeah, Linux is great... (Score:2)
Benchmark implementations in question use SAP DB (Score:5, Interesting)
They're looking for contributors to help port the tests over to other databases, and I for one will be very interested in seeing how the various database engines for Linux stack up against each other when tested using these tools.
Re:Benchmark implementations in question use SAP D (Score:1, Insightful)
Re:Benchmark implementations in question use SAP D (Score:2)
Technically speaking, yes. But, if you read the section where they state the differences between this benchmark and the "real" TPC-C, they admit that they don't actually check that the database is ACID. So you could theoretically run MySQL against this benchmark. I haven't looked at the code, but if this benchmark has only been run on the SAPDB database before, it probably is more work porting to a new database than the authors care to admit.
Come to think of it, it would be interesting to see if you could run the benchmark with a bogus null database, and see what the speed-of-light performance is.
Re:Benchmark implementations in question use SAP D (Score:1)
Actually, the DBT-1 kit has a mode where it fakes the database transaction part. The other two kits lack that at the moment.
Re:Benchmark implementations in question use SAP D (Score:1)
Re:Benchmark implementations in question use SAP D (Score:2)
I use them anyway, and have done for 2 years now.
Re: (Score:2)
Re:Benchmark implementations in question use SAP D (Score:2)
Maybe those funding the benchmarking should throw some money at the MySQL people to make their transactions pass the test. I think it would increase the performance over SAP DB incredibly.
Re: (Score:2)
Re:Benchmark implementations in question use SAP D (Score:1)
Re:Benchmark implementations in question use SAP D (Score:1)
osdldbt-general@lists.sourceforge.net
osdl_per
Re:Benchmark implementations in question use SAP D (Score:1)
Granted, we did use BEGIN as the keyword to indicate the start of a transaction. This syntax is not ANSI SQL 92 conformant.
However, in version 4.0.10 we introduced the comformant START TRANSACTION keyword as a synonym for BEGIN.
Re:Benchmark implementations in question use SAP D (Score:2)
The reason is that MySQL simply couldn't run the TPC benchmarks - its SQL parser isn't sophisticated enough. SAP DB is feature-comparable to Oracle 7 - it's a real, commercial-grade database engine. Of course, I have to wonder why they don't simply use Oracle 9 on Linux to run the benchmark.
Re:Benchmark implementations in question use SAP D (Score:1)
Re:Benchmark implementations in question use SAP D (Score:2)
Re:Benchmark implementations in question use SAP D (Score:3, Informative)
The real issue is that one wrong setting in init.ora can make orders of magnitude difference in performance. Set sort_area_size too small, and you'll be hitting the temp tablespace on disk for every sort, set your shared pool to large and latch contention will wipe out performance. Oracle are simply scared that a misconfigured database will benchmark poorly (and to be fair, it will). A database like MySQL is so simple that it's almost impossible to mistune it.
Oracle do release benchmarks when it suits them, tho', but at the moment, they're really obsessing on the performance of 9i Application Server versus WebLogic and WebSphere.
Re:Or (Score:2)
Microsoft has TPC benchmarks of their own... (Score:3, Offtopic)
Windows 2003 & SQL Server 2000 is the fastest 32-way OLTP server in the world [microsoft.com]
Unfortunately, I'm too stupid to spot any holes in the testing, but that's what you guys are for.
Re:Microsoft has TPC benchmarks of their own... (Score:2)
Who knows, may they don't bother with bothersome checksums that would only inhibit performance, or maybe it really is faster.
Xix.
TPC? TPS... (Score:1)
Enterprise Linux (Score:5, Informative)
The more benchmarks showing Linux seriously competing against other commercial OSes the better.
TPC-C benchmarks typically cost millions of dollars to run because of the expensive auditing that is required.
This tool lets any business or end user quickly gauge the performance of their servers for free, who could ask for anything more.
Clarifications (Score:1)
The TPC tests are designed, implemented and verified to enable comparisons between different systems (System A to System B comparisons) the DBT tests are engineered for working in an environment where you are trying to find out if your changes to a single system have made it faster or slower.