Biggest Technology Trend for #2020

 www.keyvaluestore.info   www.keyvaluestore.wik

For 25 years I was blessed to have worked for EMC.

They innovated and invented the category of Enterprise Storage. 

It seems silly now but I was in the room when CEO Mike Ruettgers explained (as Copernicus had done centuries beforehand explaining that it was in fact the Earth that revolved around the sun) that IT had it all wrong---servers (then made by HP, Sun, and IBM) were not the center of the IT universe, but rather your information (and where it was protected and stored) was.

We were incredible at making, selling, and installing Block storage…at a time when the world needed block storage that protected the ERP and CRM environments of the 90’s for data that fit snugly into rows and columns.

Customers wanted Performance, Consistent Performance, and for it to Scale…

Enter 2000, and the advent of the WWW created a new guard: Google, LinkedIN, FaceBook, and AWS all drove an unstructured data explosion in the same way that Oracle and SAP, etc had driven databases in the 90s.

The hyperscalers all thrived in a way that served billions instead of millions and given that block could not scale those unstructured file growth demands.

So Google, LinkedIN, FaceBook, AWS all based their file system on Key Value Store and Key Value Stores.

The need for block storage via the hyper-scalers diminished ---No maps, no dedupe, no cache!

Given the success of the hyper-scalers, a few newer entrants showed up in the market trying to emulate the success and how they created their file systems.

The only downside was that since flash was not data center ready at that time, unfortunately their technology was based on File on Block architecture since spinning disk drives were still the norm at that time.

A company in Sunnyvale came out with Scale Up via filers. It allowed customers to store unstructured data in appliances and add more appliances to that same file system.

The only downside was that since it was based on File on Block technology it hit a performance limit as you scaled.

Years later, a company in Seattle developed Scale Out technology to address that concern. It was great for adding throughput so long as you HAD to add capacity to get performance or throughput.

For today’s modern Internet of Things challenges what if you could start fresh with a clean sheet of paper and leverage all the knowledge of the Flash data center and cloud native architecture to create a new file system without the inherent architectural drawbacks of either Scale Up or Scale Out? It combine the benefits of both Scale Up and Scale Out.  You could Scale Through.

Scale Through (as defined by ESG) would have to meet four criteria that provide great value commiserate with the NVMeOF generation:

1)      It has to be Software Defined, so Commodity Off The Shelve hardware can be utilized and ride Moore’s Law to the best of breed

2)      Parity between the Reads and Writes, no longer waiting for a de-staging process due to an imbalance of reads to writes.

3)      Architected to use maximize throughput from leveraging multiple Key Value Stores as the hyper-scalers have done

4)      Decoupling performance from capacity so as your needs change, one file system can address both attributes, and back

What is possible for use cases with this Scale Through file system and high performance infrastructure?

?  Industrial Internet of Things could progress across #Smart Factory and move semi-autonomous drive to fully autonomous driving improving commutes and delivery efficiencies for millions daily

?  Hollywood could work with 32 uncompressed 4k streams instead of only 4-5 that are the norm for post-production editing, to get to the next blockbuster sooner, or add volumetric capture to features

?  In the emerging Life Sciences field, genomes which today are maxed out at 180, can be ramped up to 550 or more daily to assist gene scientists to better pursue a cure to cancer and other genetic challenges

As processing, network and Flash development has progressed into #NVMeOF over the last 18 years, so should file systems begin to #ScaleThrough!