Performance Consistency

Our performance consistency test explores the extent to which a drive can reliably sustain performance during a long-duration random write test. Specifications for consumer drives typically list peak performance numbers only attainable in ideal conditions. The performance in a worst-case scenario can be drastically different as over the course of a long test drives can run out of spare area, have to start performing garbage collection, and sometimes even reach power or thermal limits.

In addition to an overall decline in performance, a long test can show patterns in how performance varies on shorter timescales. Some drives will exhibit very little variance in performance from second to second, while others will show massive drops in performance during each garbage collection cycle but otherwise maintain good performance, and others show constantly wide variance. If a drive periodically slows to hard drive levels of performance, it may feel slow to use even if its overall average performance is very high.

To maximally stress the drive's controller and force it to perform garbage collection and wear leveling, this test conducts 4kB random writes with a queue depth of 32. The drive is filled before the start of the test, and the test duration is one hour. Any spare area will be exhausted early in the test and by the end of the hour even the largest drives with the most overprovisioning will have reached a steady state. We use the last 400 seconds of the test to score the drive both on steady-state average writes per second and on its performance divided by the standard deviation.

Steady-State 4KB Random Write Performance

The SM2260 sample has relatively poor steady-state random write performance given that it uses NVMe and MLC NAND. The use of SLC caching and the lower than normal spare area of this drive both contribute to poor steady-state performance but may not significantly impair short-term performance.

Steady-State 4KB Random Write Consistency

The SM2260 sample has an even lower consistency score than the Intel SSD 600p, which uses basically the same controller and TLC NAND, but has substantially more spare area.

IOPS over time
Default
25% Over-Provisioning

After the very short initial burst of great performance around 140k IOPS, the SM2260 sample transitions abruptly to a steady state that it maintains throughout the rest of the test with no long-term shifts in behavior. With extra overprovisioning reserved, there's an intermediate phase consisting of mostly performance around 80k IOPS and second burst at 140k IOPS before a higher performance but no more consistent steady state is reached.

Steady-State IOPS over time
Default
25% Over-Provisioning

Looking more closely at the steady state, the SM2260 sample is mostly shifting between four performance levels, with the most common being around 5k IOPS. However, when it goes through periodic phases of lower performance, it is stuttering hard and will often go for an entire second without completing any I/O. This is clearly poorly-managed garbage collection, possibly exacerbated by thermal throttling and definitely suffering from insufficient spare area.

With more overprovisioning, the severe stuttering is all but eliminated and the normal performance range jumps to around 24k IOPS with periods where it drops to around 6k IOPS.

Introduction AnandTech Storage Bench - The Destroyer
POST A COMMENT

27 Comments

View All Comments

  • BrokenCrayons - Friday, February 17, 2017 - link

    It seems like power consumption and consistency are both problems. Consistency can probably be addressed through tuning and optimization of firmware, but power consumption is probably something that can't be fine-tuned away. NVMe drives aren't as power friendly as SATA, but the inefficiencies of some recently reviewed SSDs here on Ananatech certainly seems to throw a spotlight on the problem. Between that and thermal throttling, the NVMe storage picture isn't very rosy just yet. Reply
  • KAlmquist - Friday, February 17, 2017 - link

    Even with 25% overprovisioning, I see a 1 second interval at the 3350 mark where the IOP's fall to zero. Fortunately, the performance consistency of this SSD seems more in line with its competitors on the trace-based benchmarks (The Destroyer, etc.).

    The sequential read performance at queue depth 1 is lower than that of budget SATA SSD's like the MX300. At higher queue depths, the sequential reads are faster than anything a SATA SSD can manage, but queue depths of 1 are not uncommon in a desktop environment.
    Reply
  • vladx - Saturday, February 18, 2017 - link

    Now we can see why Micron cancelled its' NVMe SSD using the SM2260, it's simply not competive. Reply
  • Drumsticks - Saturday, February 18, 2017 - link

    So what's a good roundup conclusion on the current state of NVMe drives? If we want to pick one up this year, where does the performance go from "bad for NVMe" to "not Samsung but still pretty good!"? Plextor? Reply
  • vladx - Saturday, February 18, 2017 - link

    Indeed Plextor M8Pe seems to be the best buy option at the moment in the NVMe space. Reply
  • Drumsticks - Saturday, February 18, 2017 - link

    That's what I was thinking. $220/$240 for the 500GB model right now. So much more than it used to be. Do we have any news on whether anything constraining nand supply will be alleviating any time soon? Reply
  • vladx - Saturday, February 18, 2017 - link

    The supply problems most likely will be gone after Chinese companies will flood this market in 2018. But it could happen sooner than that depending on how fast will Toshiba/Western Digital will be rolling up their own 3D NAND supply. Reply
  • Chaser - Saturday, February 18, 2017 - link

    "and the Plextor M8Pe is very slightly faster." Reply
  • AbRASiON - Sunday, February 19, 2017 - link

    If this is CHEAPER than an MX300 it's an incredibly good bargain. The MX300's are really really well priced. I'd consider it if it's dead, dead cheap.
    The 960 Evo / Pro is a joke, sure it's fast but in real world usage, it's ridiculously expensive.

    Make the 1TB version of this for $250 US or less and I'm very much in. (I doubt it will though)
    Reply
  • StrangerGuy - Sunday, February 19, 2017 - link

    I know right, Techreport tested the 960 Evo/Pro are both so close to yesteryears SATA3 SSDs in real-world load times that they are easily within the margin of error. I don't care about how NVMe is able to reach uber fast raw speeds that has zero use to me or to 99% of the people out there, but a hypothetical SSD with half the raw transfer speed of current SATA3 SSDs for 2x the capacity per dollar? That's a real bargain. Reply

Log in

Don't have an account? Sign up now