How cool is NVMe ? Part 1 - Snark Hunting

There was an interview with the CEO of a notable storage company who made a rather odd assertion when asked about their repeated rhetoric that you won’t get a good result by putting SSD into anything that wasn’t built specifically for flash. I was pretty sure that particular claim had been debunked both technically, and in the market, but, I suppose if you’ve got a marketing claim that has served you well in the past, it seems that some people, especially the less technically savvy will keep repeating it, as if “what I say three times is true” works for marketing, as much as it does for hunting snarks.

Speaking of snarks, the main reason I wrote this, which is a much abbreviated version of my original very long and very snarky reply to the whole interview, was because of the following statement.

Contradictory Claims

“First, the way they use flash as their devices get bigger, they have no path to use 5-TB or 10-TB SSDs because they perform like mechanical disk inside their designs”

when I read that, I did a wait .. what ? NetApp has been shipping 15TB drives for ages now, and much to my enjoyment, the journalist, covered that off in pretty well in the next question where he asked

You mentioned NetApp not being able to add the latest SSDs to its systems. Wasn't NetApp the first to start shipping the high-capacity 15-TB SSDs?

Yeah .. right on brother .. tell it like it is !!! To which the CEO came out with this little gem

“The problem isn't that they can't ship large-capacity SSDs. The problem is, in their system, those large SSDs perform like mechanical disks because they're still using the same SCSI and SAS”

So, first NetApp “has no path” for large drives, and then it’s well ok so they do have them but it doesn’t count because “they perform like mechanical disk inside their designs” which is because “they're still using the same SCSI and SAS”

Now, never mind that this particular manufacturer sells the vast majority of its capacity using SAS attached SSDs that use the SCSI protocol, and so is effectively damning themselves with this justification, just let me say this before I go on -

Solid State - It's all about latency !

There is no “mechanical disk” that will give you the kinds of 200 microsecond latencies that we see in a modern HA pair based solid-state array. The 15TB drives in an ONTAP based array perform just like the flash drives do in any well designed all flash array. In fact, drive for drive they often give you significantly better performance than the systems shipped by that company whose CEO was making these odd and somewhat contradictory claims.

The problem of IO density with large SSDs

OK, so to be fair, anyone that can start and build a billion-dollar storage company is clearly smart, hardworking and talented, but might not be a technical subject matter expert, and I think that what that CEO was trying to get at is the issue of I/O density. So, I’m going to throw him a bone here and explain the issue of IO density correctly.

Back 10 years ago, the average enterprise class FC mechanical disk size was about 128GB and it spun at around 15K RPM. Conventional wisdom said it could get around 250 IOPS

250 IOPS / 128GB = about 2 IOPS per Gigabyte

Today a 15TB drive can do about 100,000 random 4K read IOPS per drive which is about 400 times better than that old drive. BUT it’s also about 120 times bigger. So, per drive, you’re still three times better off. But, what happens when by using advanced data services like dedupe and compression and compaction, and you pack three times as much logical data onto that same disk ? Logically its now a 45TB disk, and your I/O density is about the same as it was from 10 years ago ! From this perspective, rather than getting 400x better performance you’re doing more or less the same. If you extrapolate this to the next generation of 40TB and 100TB drives your I/O density starts looking like an old fashioned 500GB SATA drive .. eeeew !!

Of course, that’s a little trite, because on a 15K RPM mechanical disk doing 250 IOPS you would expect got get about 7 - 10 milliseconds of latency, however with a SSD you are doing those IOPS at around 700 microseconds of latency, so things still seem at least ten times better (an order of magnitude even when the SSDs are really busy, at more normal utilisation levels those IOPS happen at about 200 microseconds). The only real disadvantage you face is that once you get really close to the 100% busy mark you hit the “knee of the latency curve” hard and your performance suffers.

Up to that point though, everything is awesome, and that’s partly why having a first rate quality of service capability on an all flash array is so important, but that’s a discussion for a different time.

Hopefully that’s addressed the first part of the “performs like a mechanical disk” issue, which has very little to do with SCSI command sets or SAS cables, but wait ... there's more! In my next post, I’ll go through the implications of exactly what happens when you connect these very large and very fast devices via NVMe and compare that to old fashioned SAS and SCSI, and maybe bring a little perspective on the hype surrounding NVMe.

Let the Snark hunt begin :-)

Other Posts in this series

• How cool is NVMe ? Part 1 - Snark Hunting
• How cool is NVMe ? Part 2 -> Throughput
• How Cool is NVMe ? - Part 3 - No waiting in Queues
• How cool is NVMe ? – Part 4 – CPU and Software Efficiency
• How cool is NVMe – A summary of benefits for external storage arrays

Part 3 -