SSD versus Enterprise SAS and SATA disks
by Johan De Gelas on March 20, 2009 2:00 AM EST- Posted in
- IT Computing
Energy Consumption
For our performance testing we used a 3.3GHz (120W TDP) Core 2 X5470; we admit to being a bit paranoid and we wanted the CPU to have plenty of processing power in reserve. In the case of purely storage related tasks, the CPU never achieved more than 15% CPU load with software RAID. Only SysBench was capable of pushing it up to 80%, but if we want to measure the power consumption of our SC-836TQ storage enclosure the SysBench value is unrealistic. In most cases, the server will run the database and perform the transactions. The storage enclosure attached to the server will perform only the I/O processing. Therefore we measure the power consumption of our storage enclosure using IOMeter, and we use a more sensible (80W) 2.5GHz Core 2 E5420 CPU. High performance enclosures (such as those of EMC) also use Xeons to perform the I/O processing.
The SC-836TQ uses one Ablecom PWS-902-1R 900W 75A power supply, one Xeon E5420 "Harpertown", 4x2GB 667MHz FB-DIMM, and one Adaptec 5085 RAID controller. "Full Load" means that the storage enclosure is performing the IOMeter Random Read/Write tests. The difference between sequential reads and random writes is only a few watts (with both SSD and SAS).
| Drive Power Consumption | ||||||
| Idle | Full Load | Idle (Drives Only) |
Full Load (Drives Only) |
Idle (per Drive) |
Full Load (per Drive) |
|
| 8 x SSD X25-E | 257 | 275 | 6 | 24 | 0.75 | 3 |
| 4 x SSD X25-E | 254 | 269 | 3 | 18 | 0.75 | 4.5 |
| 8 x SAS (Seagate) | 383 | 404 | 132 | 153 | 16.5 | 19.125 |
| 4 x SAS (Seagate) | 316 | 328 | 65 | 77 | 16.25 | 19.25 |
| No disks at all (One system disk) |
251 | n/a | n/a | n/a | n/a | n/a |
While the Intel SLC X25-E consumes almost nothing in idle (0.06W), the reality is that the drive is attached to a RAID controller. That RAID controller consumes a little bit of energy to keep the connection to the idle drive alive. Still, the fact that eight SLC drives need 129W less power than eight SAS drives while offering 3 to 13 times better OLTP performance is a small revolution in storage land.
Let us do a small thought experiment. Assume that you have a 100GB database that is performance limited. Our SysBench benchmark showed that eight SLC X25-E drives perform at least three times (up to 13 times) better than ten 15000RPM SAS drives. You need at least 30 SAS drives to achieve the same performance as the SSDs. We'll ignore the fact that you would probably need another enclosure for the 30 drives and simply look at the costs associated with an eight SLC SSD setup versus a 30 drive 15000RPM SAS setup.
We base our KWh price on the US Department of Energy numbers which states that on average 1 KWh costs a little more than 10 cents[2]; the real price is probably a bit higher, but that's close enough. It is important to note that we add 50% more power to account for the costs of air conditioning for removing the heat that the disks generate. We assume that the drives are working eight hours under full load and 16 under light load.
| TCO Comparison | |||
| X25-E | SAS 15000RPM | Comment | |
| Power per drive | 1.5 | 17.375 | 16 hours idle, 8 hours full load |
| years | 3 | 3 | |
| KWh per drive (3 years) | 38.88 | 450.36 | 360 days, 24 hours |
| Number of drives | 8 | 30 | Based on SysBench performance measurements |
| Total KWh for disks | 311.04 | 13510.8 | |
| Cooling (50%) | 155.52 | 6755.4 | to remove heat from array |
| . | |||
| Total KWh in datacenter | 466.56 | 20266.2 | disks power + cooling |
| Price per KW | $0.10 | $0.10 | |
| Total Power costs (3 years) | $46.656 | $2026.62 | |
| TCA | $6400 | $6000 | Eight 64GB SLC drives at $800 Thirty 15000RPM SAS drives at $200 |
| . | |||
| Savings | $1579.964 | ||
If you use six drives for the RAID 10 data LUN (two drives for the logs), you need the 64GB SLC drives. That is why we use those in this calculation. Note that our calculation is somewhat biased in favor of the SAS drives: the SLC drives probably run at idle much more than the SAS drives, and it is very likely that even 30 SAS drives won't be able to keep with our eight SSDs. Even with the bias, the conclusion is crystal clear: if you are not space limited but you are performance limited, SSDs are definitely a better deal and will save you quite a bit of money as they lower the TCO.
Facebook
Twitter
RSS
67 Comments
View All Comments
JohanAnandtech - Friday, March 20, 2009 - link
If you happen to find out more, please mail me (johan@anandtech.com). I think this might have to do something with the inner working of SATA being sligthly less efficient than the SAS protocol and the Adaptec firmware, but we have to do a bit of research on this.Thanks for sharing.
supremelaw - Friday, March 20, 2009 - link
Found this searching for "SAS is bi-directional" --http://www.sbsarchive.com/search.php?search=parago...">http://www.sbsarchive.com/search.php?se...&src...
quoting:
> Remember, one of the main differences between (true) SAS and SATA is
> that in the interface, SAS is bi-directional, while SATA can only
> send Data in one direction at a time... More of a bottle-neck than
> often thought.
Thus, even though an x1 PCI-E lane has a theoretical
bandwidth of 250MB/sec IN EACH DIRECTION, the SATA
protocol may be preventing simultanous transmission
in both directions.
I hope this helps.
MRFS
supremelaw - Friday, March 20, 2009 - link
NOW!http://www.dailytech.com/article.aspx?newsid=14630">http://www.dailytech.com/article.aspx?newsid=14630
[begin quoteS]
Among the offerings are a 16GB RIMM module and an 8GB RDIMM module. The company introduced 50nm 2Gb DDR3 for PC applications last September. The 16GB modules operate at 1066Mbps and allow for a total memory density of 192GB in a dual socket server.
...
In late January 2009, Samsung announced an even higher density DRAM chip at 4Gb that needs 1.35 volts to operate and will be used in 16GB RDIMM modules as well as other applications for desktop and notebook computers in the future. The higher density 4Gb chips can run at 1.6Gbps with the same power requirements as the 2Gb version running at 1066Mbps.
[end quote]
supremelaw - Friday, March 20, 2009 - link
http://techreport.com/articles.x/16255">http://techreport.com/articles.x/16255The ACARD ANS-9010 can be populated with 8 x 2GB DDR2 DIMMS;
and, higher density SDRAM DIMMS should be forthcoming from
companies like Samsung in the coming months (not coming years).
4GB DDR2 DIMMs are currently available from G.SKILL, Kingston
et al. e.g.:
http://www.newegg.com/Product/Product.aspx?Item=N8...">http://www.newegg.com/Product/Product.a...tem=N82E...
I would like to see a head-to-head comparison of Intel's SSDs
with various permutations of ACARD's ANS-9010, particularly
now that DDR2 is so cheap.
Lifetime warranties anybody? No degradation in performance either,
after high-volume continuous WRITEs.
Random access anybody? You have heard of the Core i7's triple-
channel memory subsystems, yes? starting at 25,000 MB/second!!
Yes, I fully realize that SDRAM is volatile, and flash SSDs are not:
there are solutions for that problem, some of which are cheap
and practical e.g. dedicate an AT-style PSU and battery backup
unit to power otherwise volatile DDR2 DIMMs.
Heck, if the IT community cannot guarantee continuous
AC input power, where are we after all these years?
Another alternative is to bulk up on the server's memory
subsystem, e.g. use 16GB DIMMs from MetaRAM, and implement
a smart database cache using SuperCache from SuperSpeed LLC:
http://www.superspeed.com/servers/supercache.php">http://www.superspeed.com/servers/supercache.php
Samsung has recently announced larger density SDRAM chips,
not only for servers but also for workstations and desktops.
I predict that these higher density modules will also
show up in laptop SO-DIMMs, before too long. There are a
lot of laptop computers in the world presently!
After investigating the potential of ramdisks for myself
and the entire industry, I do feel it is time that their
potential be taken much more seriously and NOT confined
to the "bleeding edge" where us "wild enthusiasts"
tend to spend a lot of our time.
And, sadly, when I attempted to share some original ideas and
drawings with Anand himself, AT THE VERY SAME TIME
an attempt was made to defame me on the Internet --
by blaming me for some hacker who had penetrated
the homepage of our website. Because I don't know
JAVASCRIPT, it took me a few days to isolate that
problem, but it was fixed as soon as it was identified.
Now, Anand has returned those drawings, and we're
back where we started before we approached him
with our ideas.
I conclude that some anonymous person(s) did NOT
want Anand seeing what we had to share with him.
Your comments are most appreciated!
Paul A. Mitchell, B.A., M.S.
dba MRFS (Memory-Resident File Systems)
masterbm - Friday, March 20, 2009 - link
Still looks like their is bottleneck in raid controller in 66% read and 33% write test. The one ssd two ssd comperseionmcnabney - Friday, March 20, 2009 - link
The article does seem to skew at the end. The modeled database had to fit in a limited size for the SSD solution to really shine.So the database size that works well with SSD is one that requires more space than can fit into RAM and less than 500GB when the controllers top-out. That is a pretty tight margin and potentially can run into capacity issues as databases grow.
Also, in the closing cost chart there is no cost per GB per year line. The conclusion indicates that you can save a couple thousand with a 500GB eight SSD solution versus a twenty SAS solution, but how money eight SSD servers will you need to buy to equal the capacity and functionality of one SAS server.
I would have liked to see the performance/cost of a server that could hold the same size database in RAM.
JohanAnandtech - Friday, March 20, 2009 - link
"So the database size that works well with SSD is one that requires more space than can fit into RAM and less than 500GB when the controllers top-out. That is a pretty tight margin and potentially can run into capacity issues as databases grow. "That is good and pretty astute observation, but a few remarks. First of all, EMC and others use Xeons as storage processor. Nothing is going to stop the industry from using more powerful storage processors for the SLC drives than the popular IOP348.
Secondly, 100 GB SLC (and more) drives are only a few months away.
Thirdly, AFAIK keeping the database in RAM does not mean that a transactional database does not have to write to the storage system, if you want a fully ACID compliant database.
mikeblas - Friday, March 20, 2009 - link
I can't seem to find anything about the hardware where the tests were run. We're told the database is 23 gigs, but we don't know the working set size of the data that the benchmark touches. Is the server running this test already caching a lot in memory? How much?A system that can hold the data in memory is great--when the data is in memory. When the cache is cold, then it's not doing you any good and you still need to rely on IOPS to get you through.
JohanAnandtech - Friday, March 20, 2009 - link
http://it.anandtech.com/IT/showdoc.aspx?i=3532&...">http://it.anandtech.com/IT/showdoc.aspx?i=3532&...Did you miss the info about the bufferpool being 1 GB large? I can try to get you the exact hitrate, but I can tell you AFAIK the test goes over the complete database (23 GB thus), so the hitrate of the 1 GB database will be pretty low.
Also look at the scaling of the SAS drives: from 1 SAS to 8, we get 4 times better performance. That would not be possible if the database was running from RAM.
mikeblas - Friday, March 20, 2009 - link
I read it, but it doesn't tell the whole story. The scalability you see in the results would be attainable if a large cache were not being filled, or if the cache was being dumped within the test.The graphs show transactions per second, which tell us how well the database is performing. The charts don't show us how well the drives are performing; to understand that, we'd need to see IOPS, among some other information. I'd expect that the maximum IOPS in the test is not nearly the maximum IOPS of the drive array. Since the test wasn't run for any sustained time, the weaknesses of the drive are not being exposed.