Why Infrastructure Automation has evolved to become a critical Business Enabler.

For years I worked in companies that developed software to monitor systems and network performance, monitor applications, manage workloads dynamically, orchestrate DevOps pipeline activity and deliver value stream management. To be successful the one thing all these tools needed was an understanding of what the infrastructure was doing and why. The problem is most provided a snapshot or a limited view of an infrastructure type, but none could accurately monitor and track the full infrastructure stack, the infrastructure across many locations, the real-time changes or understand how applications were orchestrating the changes.? This is why I joined a company aimed at solving the problem of modern infrastructure automation.

While production workloads are relatively predictable, the same cannot be said of pre-prod.? So, while IT Ops continued to use release automation and configuration tooling, DevOps practices accelerated the adoption of Infrastructure-as-Code (IaC) to support prototyping, coding, testing and the release of code as part of the CI/CD software delivery lifecycle.?

The combination of cloud, DevOps and IaC empowered developers to spin up infrastructure instances in less time it once took to open a request ticket.?? No more waiting and a major inhibitor to innovate removed.? So, while this advance accelerated the release of new and updated applications, the ability for large organizations to keep track on what the cloud infrastructure was being used for and why became a nightmare.? For most the adoption of IaC was a bottoms up decision because, at the time, IaC was primarily open source and didn’t require a purchase order.? The use of IaC was not tightly governed or controlled resulting in IaC code as unique as the person who created it.? While some may argue that teams used IaC in a consistent way it is unlikely this is the case across different development teams or across the different teams involved in software deliver pipelines.

?The challenges of using IaC are many.? Being a great developer does not always mean being a great creator of infrastructure.? This typically results in a hybrid role that has both coding and infrastructure skills. A highly sought after mix.? IaC is not something IT ops find easy to use so this typically results in either another set of infrastructure automation tooling or access to the same hybrid code and infrastructure skills.? This is why infrastructure automation has rapidly evolved with technology that automatically discovers, subsumes and leverages new and existing IaC, configuration and container tooling, normalizing the many files types and variations to simplify the formats, creating re-usable building blocks with consistent tagging, use-case alignment, governance and policies.? This new infrastructure automation acts as a control plane taking the building blocks to create environments, a full stack solution, with everything an application needs at any point in the lifecycle. By supporting older infrastructure tooling reduces technical debt while ensuring existing skills are retained with no new ones needed.? Making infrastructure available as a service to a broad range of skills is critical so providing it as each person needs is key whether through a self-service catalog or integrated into an IDP as a set of services to improve the developer experience.

So, with all that said what is the result? With infrastructure complexity reduced it still isn’t obvious how infrastructure automation becomes a business enabler. The answer is not just about removing complexity or aligning the infrastructure usage with use-case purpose - but also market timing.? When the market was healthy and top-line growth was the focus, it was easy for the business to remain oblivious to infrastructure complexities and the actual cost.? Cloud was strategic, easy to attain with value assumed. Then the market started to tank. Uncertainty crept in, questions on cloud cost started to be asked.? Why don’t we know what our cloud usage is today? Why does it take so long to understand how the ‘cloud account’ costs map to business value?? Some assumed the combination of the cloud providers account-based usage tracking and FinOps tools providing a way to understand costs to negotiate the best deal is enough. They were wrong. According to analysts and the FinOps foundation surveys, cloud costs remain a major issue - on par with security and privacy concerns.?? The fact is cloud usage remains hard to track, it constantly changes, it can be ephemeral and as a result cloud costs change in real-time.?? No matter how hard companies try to force the creators of infrastructure to standardize tagging or use tools that ignore tagging and dictate their own proprietary methods the problem continues to grow.?

This is where infrastructure automation and cloud cost management collide.? With a modern infrastructure automation platform, as described above, cost is factored in as part of the creation and provisioning activity.? When, for example, a cloud instance is needed the consumer is shown choices with the initial cost. This allows choices of infrastructure and cloud provider to be made. If the cost is too high and exceeds a budgetary policy the consumer will either receive a warning or prevented from choosing the selected environment, with other lower cost options provided.? Once deployed, the costs are tracked consistently because the platform has normalized automation file disparity to deliver a standard view of cost.? While a FinOps practice waits for cloud invoice-based lagging indicators to understand cost, the modern infrastructure automation platform provides full cloud usage reports in minutes. However, because each environment is associated with a use-case purpose the value delivered by the cloud is understood immediately allowing the platform to allow rapid and accurate cloud optimization decisions to be made.? Traditionally this can typically take weeks of analysis trying to work out who was using the cloud for what purpose (each cloud account can have hundreds of cloud instances of varying importance).? As the modern infrastructure automation platform has day 2 activities built in there is no chance under-utilized, dormant or zombie cloud instances will be left running automatically reducing significant costs caused by waste.?? Lastly, the modern infrastructure automation platform can easily associate different cloud instances with a common purpose - even across different clouds.? This means a business can get a consolidated view of cloud usage and cloud cost for pipelines, applications, projects, teams or just an individual. ?

Infrastructure is getting more complex; skills are already creating a bottleneck and cloud costs continue to increase while being increasingly scrutinized.? For those of you who managed to get to this point it would be interesting to get your thoughts. If you would like to understand what a modern infrastructure automation platform can do, please visit www.quali.com where you will be able to understand and trial (for free) our Torque platform (providing a normalized view of everything you use to provision cloud in minutes) and already delivering modern infrastructure automation to over 1 million users.