More RAID 5

The Random 66% read + 33% write is still the most interesting scenario, as it mimics an OLTP database.

RAID 5 IOMeter Random 66% Read and 33% write

Again, we get some weird results. We suspected that the cause of this phenomenon was different from the previous benchmark. With eight drives, for each write the RAID controller has to perform seven additional reads to calculate the parity block, which makes writes more costly than with four drives.
 
Update:  Made the wrong assumption here. A read of the parity block and one other drive is enough, you don't need 7 reads. So two reads followed by 2 writes are enough for an 8 drive RAID-5. Thanks to Tobias for pointing this out. So we are not completely sure where this performance decreases (4 to 8 drives) comes from. 
 
Once again, we tried out the Areca 1680 to provide us with more insight.

While it is true that the Adaptec shows some bad scaling in RAID 0, both controllers fail to show any decent performance increase with more than four disks in RAID 5. This only confirms our suspicion: the SLC drives are capable of reading and writing at blazing speed, so the RAID controller is swarmed by the massive amount of reads per second that need to happen for each write. As the amount of reads for one write that need to happen on the eight drive array is more than twice the amount on the four drive array, performance decreases. This is not an issue with the magnetic disks, as they are performing the same task at much lower speeds and thus require much less reads per second. We may assume that the RAID controller is holding back the SSD array, but proof is better than conjecture. We attached the four drives to each controller and made the Xeon at 3.33GHz the RAID controller (i.e. software RAID in Windows 2003):

RAID 5 with Eight X25-E SSDs
  One IOP348 controller and
Hardware RAID
Two Controllers and
Software RAID
Sequential Read 688 MB/s 1257 MB/s
Sequential Read/Write 221 MB/s 257 MB/s
Random Read/Write 169 MB/s 235 MB/s

Once we use the Xeon as the storage processor, the bottleneck is gone (or at least less narrow). Sequential read performance is twice as high as with four disks, and although the scaling of random read/write performance is nothing to write home about (19%), at least it is not negative anymore.

Should we blame Adaptec? No. The bandwidth and crunching power available on our Adaptec card far outstrips the demands of any array of eight magnetic disks (the maximum of drives supported on this controller). You might remember from our older storage articles that even the much more complex RAID 6 calculations were no problem for the modern storage CPUs. However, the superior performance of the Intel X25-E drives makes long forgotten bottlenecks rear their ugly head once again.

RAID 5 in Action SQL Server and RAID 5
Comments Locked

67 Comments

View All Comments

  • marraco - Wednesday, March 25, 2009 - link

    The comparison is not fair, but can be fairer:

    If the RAID of SATA/SAS disks is restricted to the same storage capacity than the SSD, limiting the partition to the fastest external tracks/cilynders, the latency is significantly reduced, and average read/write speed is significantly increased, so

    PLEASE, PLEASE, PLEASE

    Repeat the benchmarcks, but with short stroking for magnetic disks.
  • JohanAnandtech - Friday, March 27, 2009 - link

    May I ask what the difference with the fact that we created a relatively small partition across our RAID-5 raidset? Also, you can imagine that our 23 GB database was at the outer tracks of the disks. I have to verify, but that seems logical.

    This kind of testing should give the same effects as short stroking. I personally think Short stroking can not be good for your actuator, while a small partition should be no problem.
  • marraco - Friday, March 27, 2009 - link

    See this link.
    http://www.tomshardware.com/reviews/short-stroking...">http://www.tomshardware.com/reviews/short-stroking...

    Clearly, you results are orders of magnitude than those showed on that benchmark.

    As I understand, short stroking increase actuator health, because reduces physical acceleration on the actuator.

    Anything necessary, is to use a small partition on the fastest external track.

    you utilized a raid 0 of 16 disks, with less than 1000 gb/second.

    On Tomshardware, a raid of only 4 disk achieved average (not maximun) 1400 to 1600 Mb/s. (of course, the test are not the same; for that reason, I ask for new test)

    About the RAID 5: I would love to see RAID 0.

    I are interesed on comparing a fast SSD as the intels, (or OCZ Vostro/Summit), with what can be achieved at the same cost, with magnetic media, if the partition size is restricted to the same total capacity than the SSD.

    Anyway, thanks for the article. Good work.

    So good, I want to see more :)
  • marraco - Sunday, April 5, 2009 - link

    Please, tell me you are preparing such article :)
  • JohanAnandtech - Tuesday, April 7, 2009 - link

    We are investigating the issue. I like to have some second opinions before I start heavy benchmarking on THG article. They tend to be sensational...
  • araczynski - Wednesday, March 25, 2009 - link

    wow, color me impressed. all the more reason to upgrade everything to gigabit and fiber.
  • BailoutBenny - Tuesday, March 24, 2009 - link

    Can we get any updates on the future of chalcogenide glass (phase change) based drive technologies? IBM's Millipede and other MEMS probe storage devices? Any word about Intel and STMicroelectronics' shipments of PRAM samples to customers that happened last year? What do the rumor mills say? Are these technologies proving viable? It is difficult to formulate a coherent picture for these technologies without being an industry insider.
  • Black Jacque - Tuesday, March 24, 2009 - link

    RAID 5 in Action

    ... However, it is rarely if ever used for any serious application.

    You are obviously not a SAN Admin or know too much about enterprise level storage.

    RAID 5 is the mainstay of block-level storage systems by companies like EMC.

    In addition, the article mentions STEC EFDs used by EMC. On the EMC CLARiiON line, those EFDs are provisioned in RAID 5 groups.


  • spikespiegal - Wednesday, March 25, 2009 - link

    [quote]RAID 5 is the mainstay of block-level storage systems by companies like EMC. [/quote]

    Which thus explains why in this day in age I see so many SANs blowing entire volumes and costing days of restoration when the room temp gets a few degrees above ambient.

    Corrupted RAID 5 arrays have cost me more lost enterprise data than all the non-RAID client side disks I've ever replaced; iSeries, all brands of x386, etc. EMC has a great script to account for this in which they always blame the drives first, then only when cornered by an enraged CIO will they admit it's their controllers. Been there...done that...for over a decade in many different industries.

    If you haven't been burned by RAID 5, or dare claim a drive controller in RAID 5 mode has a better MTBF than the drives it's hosting, then it's time to quite your day job at the call center in India. RAID 5 saves you the cost of one drive every four, which was logical in 1998 but not today. At least span across multiple redundant controllers in RAID 10 or something....
  • JohanAnandtech - Tuesday, March 24, 2009 - link

    I fear you misread that sentence:

    "RAID 0 is good way to see how adding more disks scales up your writing and reading performance. However, it is rarely if ever used for any serious application."

    So we are talking about RAID-0 not RAID-5.
    http://it.anandtech.com/IT/showdoc.aspx?i=3532&...">http://it.anandtech.com/IT/showdoc.aspx?i=3532&...

Log in

Don't have an account? Sign up now