Re: IOPS benchmarks - looking for opinions

On Fri, 28 Mar 2008 12:58:16 -0500, nik Simpson <nik_s@xxxxxxxxxxx>
wrote:

fx [François-Xavier Peretmere] wrote:
Hi everyone,

I recently read about the heated contest between two Storage vendors
regarding some IOPS tests one did with the other's gear. The latter's
claim is that the IOPS benchmark is biased and irrelevant, and therefore
they don't run it with their own products. Not being neither a HPC or a
high IO expert, I can't really decide whether or how relevant the claims
are - whilst I found some arguments compelling.

Being curious, I'll be interested in reading what some of the
knowledgeable minds haunting this group would have to say regarding the
IOPS benchmark validity and how relevant they are to the "real world" -
whether or not they are a good proxy to assess how a given piece of
hardware is able to sustain an heavy IO load.

I don't want to discuss the validation process used in by one of the
vendor, I have my own opinion on this matter, and I'd rather prefer to
avoid replicating the heated exchanges I've read in the respective blogs
of those two vendors :-)


For the record, I assume you are talking about the Netapp/EMC SPC-1
results that Netapp "kindly" published on EMC's behalf. This is much
more complex than simply running 512KB IoMeter type loads.

I thought Dave Hitz explained the situation very well and had good
points. But I hate EMC so my bias is unfettered.

But, to the point. Synthetic tests are just that, synthetic. They
can give you great insight into worst case or specific case IO
scenario's. Some of those may or may not apply to your workload.

Real World is so diverse that there's no Real World validation other
than each users own application.

Oracle data warehousing v. OLTP v. EDA grid apps v. web server v.
.......

You get the picture. The truth of it all is that you HAVE to know
what your target is, or at least what your current IO is. You can't
just say "build me a house" and expect to be happy with the outcome.
There are details involved.

~F
.

Relevant Pages

  • IOPS benchmarks - looking for opinions
    ... I recently read about the heated contest between two Storage vendors regarding some IOPS tests one did with the other's gear. ... The latter's claim is that the IOPS benchmark is biased and irrelevant, and therefore they don't run it with their own products. ... Hundreds of volunteers worldwide volunteering ...
    (comp.arch.storage)
  • Re: IOPS benchmarks - looking for opinions
    ... I recently read about the heated contest between two Storage vendors regarding some IOPS tests one did with the other's gear. ... The latter's claim is that the IOPS benchmark is biased and irrelevant, and therefore they don't run it with their own products. ... This is much more complex than simply running 512KB IoMeter type loads. ...
    (comp.arch.storage)
  • Re: IOPS benchmarks - looking for opinions
    ... I recently read about the heated contest between two Storage vendors ... regarding some IOPS tests one did with the other's gear. ...
    (comp.arch.storage)
  • Re: IOPS benchmarks - looking for opinions
    ... I recently read about the heated contest between two Storage vendors ... regarding some IOPS tests one did with the other's gear. ...
    (comp.arch.storage)
  • Re: IOPS benchmarks - looking for opinions
    ... I recently read about the heated contest between two Storage vendors regarding some IOPS tests one did with the other's gear. ... The latter's claim is that the IOPS benchmark is biased and irrelevant, and therefore they don't run it with their own products. ... Nor is 512 byte transfers overly useful for predicting application performance with apps that regularly use multi-megabyte transfers. ...
    (comp.arch.storage)