Planning for Infinite Capacity in Cloud Infrastructures

As more businesses migrate their compute needs to cloud service providers, the perception is there is infinite computing capacity available for consumption with "the cloud". The reality is that cloud service providers have to deal with many constraints behind the scenes to create the perception of capacity to allows be available to be consumed on demand.

I spent considerable time focused on IaaS capacity in the cloud over the last year including servers supporting IBM AIX, Red Hat Enterprise Linux and SUSE linux and Microsoft Windows. The IaaS platform provided a range of user cases including un-managed/customer managed VMs, service provider Managed Operating Systems, Managed databases, Managed middleware, Managed SAP (including SAP applications and landscapes, virtual HANA, physical HANA) and Managed Oracle just to name a subset. Each area has their own nuances which must be addressed with proper capacity planning. Rather than going into detail here, I will focus on the servers in this post.

Looking at the physical servers which host the Virtual Machines (VMs), one physical server can host a considerable number of VMs on physical servers. Physical servers have grown to nearly 100 cores and terabytes of physical memory. The likelihood of all the VMs demanding resources at the exact same time is low. By looking at the history of usage patterns, diverse workloads can be residing on the same physical host can be placed to maximize the utilization of the overall physical server across a day, week or month. However, one must consider the blast zone if a physical server hosting the VMs should fail, so it is not just about just having a few very large physical servers.

Physical servers are usually in a cluster of physical servers which allows dozens of physical hosts, which in turn allows VMs to be dynamically re-balanced across multiple physical servers. This way, as an individual VM workload bursts, the clustering software can move VMs around the cluster to handle spikes or bursts in workload from individual VMs.

Through the use of Software Defined Networking (SDN) VMs can even be moved across different physical clusters without an outage. The flexibility this provides is pretty incredible today from what was available a few years back, including the ability to move VMs between different physical data centers.

Of course there are other constraints that a service provider must deal with including: network capacity, storage capacity, power and cooling capacity within a data center and it is important to trend usage growth into the future to understand when additional physical resources or data centers need to be added to avoid an out of capacity condition.

So while the goal of a service providers is to create the perception of infinity capacity of resources in a cloud, the reality is this goal is only fulfilled through diligent tracking and planning by skill professionals.