Benchmarks: Are They Good For You?

Acquiring a server isn't a piece of cake. In fact, it's a good bet that, depending on your installation, the server will be the most expensive piece of IT equipment that you install. So it's a good idea to make sure that the server you get will do the job.

But how do you do that? There are so many servers out there (iSeries from IBM, Solaris servers from Sun, Linux servers from a host of people, Windows servers from the multitudes) that just deciding which OS you want is a chore, not to mention the question of architecture. Then there's the question of which server in the chosen architecture/OS you decide you want. Help!

You can cut through a lot of this fog, though, by using the results of some standardized performance benchmarks that are available free for the asking, over the Web.

"I advise customers, clients and analysts that benchmarks provide a common language to get relative information about one server versus another," says David Gelardi, director of IBM Systems Group Benchmarking and Design Centers." The TPC and SPEC benchmarks are all structured, and we all know the rules for the tests."

TPC (www.tpc.org) is the Transaction Processing Performance Council, a non-profit group whose members include most, if not all, server manufacturers and some software vendors as well. The group standardizes tests that result in benchmark numbers, and it keeps people from making exaggerated claims about the results of their testing. In the case of the TPC-C test, the group's flagship and probably most important benchmark for servers, the manufacturer runs the test in his own facility, but under the watchful eye of an independent auditor from TPC, who make sure the test was conducted according to the rules.