The Private Cloud is Dead

Long live the private cloud.

There has been a lot of discussion (perhaps gesticulation is a better word) this week in the wake of the announcement that Dell, a massive and privately held server company, is about to borrow a big bunch of money to purchase EMC, a massive publicly held storage (and virtualization) company.  Despite the fact that we’ve all known that some “clash of the titan”-style mergers were coming for years now, the fact that we are finally seeing one seems to have thrown the entire tech industry into a fit of existential, navel-gazing posturing around What It All Means.

Over the course of the week, I took time to read a few of these articles.  The canonical example was probably a short piece by Cade Metz at Wired.  Mr. Metz’s article, like several others I’ve seen, effectively says that the merger is the sign of the end of days for enterprise IT vendors, because it’s all going to the cloud.  Full stop.

The struggle of EMC and Dell is not the failure of enterprise IT, it is the failure of an outdated approach to building enterprise IT.  So let’s be very clear about why AWS is growing and Dell and EMC are struggling: Dell and EMC sell component solutions that hinge on a traditional 5-year capital cost amortization cycle: you pay money today against a prediction of value that you expect to attain over five years of use.  AWS is an end-to-end solution that is priced for instant gratification, as are the large number of “next generation” enterprise appliances -- like Coho Data’s -- that are eating EMC’s lunch in enterprise server rooms around the world.  Dell/EMC’s problem isn’t that the private cloud is failing, it’s that two large companies have largely failed to innovate in a way that matches the needs of the modern enterprise.

The interesting conversation that is stemming from the Dell/EMC deal is really one about trying to define and predict the future for public and private clouds. Given that this distinction is being drawn as a black and white division between past (private) and future (public), it’s enlightening to consider some of the actual similarities between Dell/EMC and AWS:

Like Dell, AWS is a logistics company.  I remember reading articles, about 10 years ago that talked about “Dell Time.”  That the real brilliance of Dell as a corporation was one of operational efficiency: unlike all other computer vendors at the time, Dell carried virtually no inventory.  They demonstrated absolute mastery of a complex and layered supply chain that was able to take orders for computers and build them, from scratch, and deliver them to customers in a matter of days.  As a result of this efficiency, Dell was able to drive greater volume, survive lower margins, and dominate competitors who struggled to manage pricing that had to absorb inventory and stocking commitments that moved slower than CPUs were revving generations.  This supply chain was central to Dell’s value, but so was the fact that, as PC clones, their systems were better integrated, better supported, and more enjoyable to use than anything else at the same price point.  

The brilliance of Amazon is that they are using the exact same playbook.  AWS has used virtualization to even further optimize the notion of an operationally hyper-efficient supply chain.  Using the only real lever in computer science -- a layer of indirection -- AWS’s VMs allow a customer to order, compose, and deliver a computer in a matter of seconds.  More importantly though, AWS bolsters the value of that virtual computer with a set of features, including managed internet connectivity, scalable storage,  and additional services that have traditionally been difficult (or at least expensive) to build and deploy in the enterprise datacenter.

So AWS’s advantage today is that of a cheaper, richer, computing device and the same was true of Dell about 10 years ago.

Like EMC, AWS is a storage company.  Here’s the rub though: the very value of on-demand virtual servers is the thing that makes it a risky business.  Cloud services are inherently a race to the bottom on pricing, and the easier it is to move between cloud vendors the more opportunity there exists for a broker to commodify those services, allowing customers to switch between them on a monthly, daily, or hourly basis.  At the end of the day, data has mass -- it’s non-trivial to move in volume -- and so data is sticky.  Data is the thing that holds a customer on a given cloud provider’s system, and pricing is set up as to encourage data (whether that data is raw business data or VM images) to be moved into the cloud at little or no cost, to charge competitively for access while it is there, and to charge discouragingly high prices to move it out.

When public cloud providers such as AWS began business, they immediately realized that the cost and scalability of existing enterprise storage would be a non-starter for their business model.  They invented scalable, and demand-based storage systems, which simply hadn’t been built before.   These systems were one of the biggest technical differentiators that environments like AWS achieved in their early days, and it is because AWS is (to a large part) a storage company that they continue to be successful today.

Dell and EMC are struggling because they haven’t innovated for the modern enterprise.  Dell and EMC have both largely failed to innovate in the way that other emerging enterprise vendors (and I include my own company here) have been for years at this point: There are many reasons for enterprises today to continue to run their own IT.  That IT may remain, as it is today, in an enterprise datacenter.  It may, alternatively, be racked in a colocation facility that offers many of the managed benefits of a system like AWS, possibly including high-speed connectivity into the AWS environment itself.  Dell and EMC aren’t symbolic of the “problem” of enterprise IT generally, they are symbolic of the legacy approach to system building that has led to hundreds of exciting enterprise startups emerging over the past 5-10 years.

Here’s one recent example that I am intimately familiar with:  Last week, Coho, Intel, and UBS AG published a reference architecture for a scalable, high-performance and multi-tenant big data environment that is converged with an enterprise storage platform.  This example is emblematic of how enterprises, at scale, are achieving incredible value and consolidation by taking advantage of the very technologies that were used to build public clouds in order to build and scale their own systems.  Multi-tenant, multi-distribution big data jobs running in isolation, in docker containers, in the storage system.  In the data.  

To me, the real value of this reference architecture is that it realizes exactly the benefit that is being touted about the public cloud: it removes the need for a technically deep and hands-on operations team that may have even written the storage system itself.  Instead, it uses a combination of commodity hardware and production-grade software to make it fast and simple to spin up analytics jobs on data.  It is about time to value, time to insight, and operational flexibility and it’s in an appliance and in the enterprise datacenter.  It’s a single example of how the enterprise itself is shifting, and learning from the operational model of cloud computing.

Cloud environments are becoming a commodity. Just over 10 years ago, AWS achieved a massive technological differentiator: the ability to build a scalable public cloud at all.  In the time that has passed, they have continued to innovate, and to put distance between the capabilities of their environment and those of traditional enterprise vendors.  I have written before about how impressed I am with the technologies, especially things like Lambda, that Amazon is building.  But let’s be honest with the fact that the innovation that has been pioneered within a few large corporate datacenters, in controlled environments like Amazon and Google, will be commodified.  As these technologies become successful and established, appliances like ours will provide similar value in environments that are owned and controlled by customers themselves.  It is clearly way easier to build a private cloud environment today than it was a decade ago, and companies like Coho are helping to make it easier every single day.  I work directly with a number of private cloud environments that number in the thousands of physical servers and they are continuing to build and run their own systems precisely because it is more cost-effective than the public cloud.

At the end of the day, infrastructure is always just a tool.  It is a distractingly false dichotomy to draw an us-or-them separation between public and private clouds.  Most large enterprises I talk to are using both, and plan to continue to use both.  The challenge for public cloud environments will be one of continuing to innovate, and to take advantage of economies of scale and fast cycle times in deploying new functionality, of staying ahead of what can be achieved just as easily in software.  More realistically, these environments will continue their race to the bottom and provide real “utility IT” for things like low-cost backup storage, burstable spot-instance compute, and full-featured virtual computing environments for customers that are small enough not to benefit from their own modest economies of scale with an appliance-based architecture in a managed co-location facility.

The challenge for enterprise appliance vendors will be in providing the same operational and economic flexibility of the public cloud, with the control and autonomy that comes with owning your own environment.  I hope that Dell/EMC are able to better realize this vision together than they were apart. I know that we at Coho have been working hard to provide exactly these capabilities for data management in private clouds since we started, and I believe the approach has an incredibly bright future ahead of it.

Long live enterprise computing, wherever it may reside.