Cloud Complexity core to Cloud Reliability Issues?

There is a lot of research indicating how the use of multicloud architectures within enterprises is growing. Indeed, a lot of companies are already multicloud, and this architecture is becoming a deliberate choice. What’s a well understood concept is that movement towards multiple clouds – and the hybrid cloud – is even a logical evolution, and one that likely to drive us into the future.  

What are the analysts saying, or said?  Going back to 2016 Forrester stated that, while organizations already use multiple clouds, they will do so even more in 2017 and beyond.   CIOs are stepping up to orchestrate the various cloud ecosystems connecting employees, customers, partners, vendors and – with the Internet of Things in mind – devices to serve rising customer expectations.

Of course, this is a macro pattern, and other research firms are taking notice.  In its Worldwide Cloud 2017 Predictions, IDC predicted that over 85 percent of enterprise IT organizations will commit to multicloud architectures by 2018.

So, what does leverage multicloud mean to me? It means that we’re adding complexity, and if we’re adding complexity we could be doing so at the cost of reliability.  The purpose of this article is to explore that concept and determine if we actually have an issue that we need to deal with if we’re moving forward with cloud computing.   

Driving to Complexity

Putting surveys aside for the moment, there are a few key issues that I see in the market today:

1. Most enterprise adoption takes place around an existing business problem, with a clear business case. Enterprises do not adopt cloud computing because it’s trendy, they need to solve real problems right out of the gate, and if those problems require that we leverage complex architectures, so be it.  

2. Most enterprise adoption involves existing application migration rather than new application development. Most enterprises that migrate applications try to do so as quickly as possible, and do not focus on “refactoring” applications to make more efficient use of native cloud services.  Thus it’s a lift and shift world, and will be for some time until we run out of applications that don’t force refactoring.   

3. Most enterprise adoption occurs around existing technology partners. This includes technologies from guys like IBM, Oracle, and VMware. New public cloud brands, such as Microsoft, Google, and AWS, are in there as well, of course. Indeed, when enterprises talk to me about their cloud needs, they typically have a laundry list of companies they have already selected as part of their path to the cloud. It’s very difficult to get them to think outside those boxes.

4. Security and governance continue to be afterthoughts. This, despite the fact that security is consistently listed as the number one priority for enterprises as they move to the cloud. Most rely on technology rather than planning to meet their security needs. However, most security solutions are ineffective unless implemented with a great deal of planning.  

5. Cost is not as much of a concern as we originally thought. While most enterprises will claim to move to cloud computing for cost efficiency reasons, the reality is that most are moving for shifts in budget. Leadership is getting tired of paying for data center upgrades and expansions each and every year, and have set deadlines for IT to find other locations to support core IT systems.  

The resulting solutions are clouds mixed with traditional systems, mixed with other outsourcing options (e.g., colos and managed services providers).  However, the mother of all mixes is the rise of multicloud architectures which means that we’re leveraging more than a single public cloud provider, and in many cases 3 or more.  This means enterprise IT must also deal with the resulting rise in complexity.

Understanding That We Have a Problem

Consider the following oversimplification of the problem: 

Note that as you might expect as the complexity rises, reliability decreases at about the same rate.   This is due to a number of different factors, including:  

1. The more links in the chain, the weaker the chain is.  If we have a single system running, that is the most reliable if not set up with redundancy.  As we add systems that are interdependent on each other reliability goes down, considering that a single system failure can bring down the entire chain of systems.  You can consider each link a single cloud provider.    

2. The more cloud services that are leveraged within a single cloud, the more likely there will be a failure.  So, we’re leveraging many public clouds, and then many cloud services on those clouds, we’re not only running into reliability issues at the multi or many cloud level, but also within each cloud in that the more services we leverage that are interdependent, and not redundant, the more we’re doing so at the expense of reliability.   

3. The more cloud and cloud services we have to keep track of, the more likely humans are going to make a mistake.  We have a tipping point when the number of clouds and cloud services become too much to track for our human brains.  While we can leverage tools such as cloud management platforms (CMPs), or cloud services brokers (CSBs) there is still a point when the complexity grows bigger than we can understand, or track.  When that happens, things are missed, and things go wrong.      

The enterprise architect in me would suggest that the best solution for enterprises that are already hindered by architectural complexity without the presence of cloud computing is to get their respective “acts together” before they adopt cloud computing. However, the world does not work that way. In the real world, most enterprises would have to do a ton of work over many years to be perfectly ready to move easily to cloud-based platforms.  

The root issue is the ability to manage complexity, including the addition of applications (new and old) that will run on public cloud platforms. The trick is to think in terms of replacement, and not additions. Applications that exist on traditional platforms (such as LAMP in the enterprise data center) should be moved in bulk to surrogates in the cloud. Then, after some acceptance testing, those platforms should be decommissioned with extreme prejudice.

Mistakes that existing enterprises make involve moving a few applications to the cloud, which creates the need to maintain applications that run on AWS or Google, while still staring at the same hardware in the data center. Nothing changes, other than things get more complex. At issue is always those one or two applications that are not migrated. Any cost savings made through the use of public cloud-based resources is quickly gobbled up by the cost and complexity of maintaining one more platform. However, this new platform happens to be within a public cloud service.

Here are a few Do’s and Don’ts for enterprises that balk at the use of cloud computing due to the complexity it may/will bring:

? Do enterprise architecture and overall IT planning. Yes, that means some advanced planning, in terms of what applications and data will run where, and why. Cloud-based platforms, at the end of the day, are nothing more than other systems that you must manage. Thus, the fewer you need to manage, the simpler the architecture, and the more likely you are to succeed longer term. These are approaches and disciplines that are already well defined, and well known.  

? Don’t push things to cloud, just for the sake of pushing it to cloud. Applications should have clearly defined benefits when running on cloud-based platforms, with the objective of migration to decommission existing platforms so architectural complexity is actually reduced, or, at least stays about the same.  

? Do consider automation. While cloud management platforms and service governance tools clearly add value, most enterprises don’t consider them early enough in the process of moving to cloud computing. They should be systemic to the architecture, and actually reduce complexity by abstracting those managed clouds away from the complexity using the “single pane of glass” type of approach to architecture.  

? Don’t stop measuring. Keep metrics in mind as you move to the use of cloud computing, including the ability to determine and measure cost efficiencies and overall IT efficiencies. Be prepared to get some disappointing numbers at first, and adjust the process, technology, and the architecture as needed.  

The purpose of this article is to explore that concept and determine if we actually have an issue that we need to deal with if we’re moving forward with cloud computing.    

If the notion we’re putting forth in this article is that there is a tradeoff between cloud reliability and complexity, in that they are inversely related, what are we to do about reality as we increase cloud complexity?  This keeping on mind that for most enterprises complex using of multicloud solutions are just a forgone conclusion.  

There are a few things to consider:  

First, leverage redundancy when you can.   Keep in mind that one of the core reasons that complex clouds that leverage multicloud have demising reliability due to the fact that there are more interdependent systems that can fail.  Thus, if you make most of them backed up with some (active-active) redundancy, meaning running standbys that are ready to take over, the reliability will likely go way up considering that interdependent system failure will be mitigated by redundancy.    

Second, leverage automation when you can.  This is a matter of abstracting yourself away from the complexity, and also automating around cloud and cloud services outages and other failures.   We’re leveraging some type of tool that’s able to automate reactions to systems reliability issues, again mitigating the effects of reliability issues since they are self-corrected using automated tools, such as cloud orchestration, CSBs, and CMPs.   

That said, there is no magical formula here, and dealing with increasing complexity is nothing new for those building systems.  Cloud computing is just another layer of technology and a different consumption model.   

What is clear, however, is that we don’t yet understand the impact of the complexity that multicloud will bring, we’re just getting the initial implementations up and running.   I suspect that we’re learn from the many mistakes that are likely to be made in the next several years, including a few that cost millions in lost revenue.  

However, if the alternative is that we’re don’t move enterprises IT forward, that actually much worse than doing nothing at all.   That will kill your business much fast than complexity ever could.