Looking Back To Predict The Future

Ever wondered what it would be like to be able to predict the future? Usually the best indicators of the future are what has happened in the past. Even if there are disruptive forces at play, get a long enough view, and I mean a really long view, and you can see these things in context too.

When I came into the Data Storage industry in the early 1990’s desk top hard drive capacities were measured in 10’s of Megabytes which you either backed up with 10’s of floppy disks or a single tape drive.

Those floppy disk drives are now consigned to museums and the hard drives are becoming an endangered species too. Why? Disruptive technology in the shape of Solid State or “Flash” drives. No moving parts, less power requirement, faster read/write times, and now capacities to rival and exceed the humble spinning disk. You see fastest wins for everyday access. But what about long term storage where speed is no longer the issue - where pure economics wins the day?

In the world of archiving, it is not the Fastest that wins but the Densest that wins ($/TB) – and here Tape still holds the day. In fact, with current trends in Tape technologies from companies like IBM, Fujitsu and Sony, tape is here, it is now and it is to stay for the foreseeable future.

https://www.theregister.co.uk/2017/08/02/sony_ibm_tape_dense/

Did I just say, “foreseeable future”? Curiously enough the current world of FLAPE (combining the speed of Flash with the economics of Tape within the enterprise) was predicted by my IBM colleague Steve Legg when I first joined IBM more than 10 years ago. Back then Steve had a chart in his customer decks showing Areal Density (bits/square inch) of Disk drives. This chart showed ever increasing densities but with a couple of interesting kinks. These “disruptions” in the story – one negative, the other positive – would have been invisible to any company getting into storage after the early 1980’s.

The first was in the early 1970’s when it suddenly became harder to gain areal density improvements. This forced users to look at other technologies and techniques to balance the demand for storage which was outstripping the ability of the storage industry to reduce its price. One of these instruments was Hierarchical Storage Management (HSM) – moving data to a lower cost medium – and another was a resurgence of tape as that lower cost medium.

With the introduction of GMR head technology in the early 1980’s, - the positive kink in the graph - we entered a period of relatively easy areal density improvements once more. This saw the rise of most of the Storage Array companies in the world today, arriving as they did without the hindsight of the intervening 25 years; without experiencing the challenges that come when areal density improvements get harder. I was selling HSM systems then and it was hard work – why invest in tape when the disk would be twice as big and half the price in 18 months’ time?

What none of us saw then, except perhaps the folks at IBM because they had seen it before, was that things were going to get tight again. According to that chart of Steve’s from 2007 we were right on the cusp of another inflection. I recall him saying that since fastest wins for access and densest wins for economics, within a few years solid state disk would be displacing spinning disk. Although Steve didn’t coin the phrase FLAPE, he did predict its appearance. Was Steve Legg a prophet or, to borrow from Sir Isaac Newton, was he merely seeing further because he stood on the shoulders of giants?

IBM is one such giant in the storage industry, because it has been there since the very beginning. When other disk array manufacturers were disparaging tape and telling clients to get rid of it in favour of spinning disk, IBM continued to invest in it as a medium of the future. Because when you have been around the track more than once, you know where the bumps are and you can be prepared for them. That’s not really predicting the future; it is just learning from the past!