Code or Console? How do you want to configure your infrastructure?

IT Infrastructure professionals have had a turbulent couple of decades. After all, it wasn't all that long ago that setting up a new server meant standing in a cold data-centre!

How the world has changed for the Infrastructure crowd! The mass-adoption of virtualisation, and then Cloud platforms, now mean that much of your infrastructure is IaaS or PaaS, deployed even more quickly, and with a great deal of configuration "out of the box".

These disruptions accelerated overall IT systems change, by templatising fulfilment, and reducing infrastructure lead times. The next disruption to hit this area is Infrastructure as Code (IaC), the practice of using software to provision and configure infrastructure in a repeatable, consistent and automated manner.

In this article I discuss IaC for those who aren't familiar, discuss which projects and systems it's best for, and how to avoid common pitfalls.

So, what is Infrastructure as Code (IaC) really?

For those who prefer a more "tangible" picture, infrastructure as code means:

1. Using code and/or configuration files to define virtual infrastructure configurations (VMs, operating systems, storage volumes, and almost any cloud components private or public)
2. The code and/or config files are just a way for the IaC tools to tell the virtualisation layer - or cloud platform - what we want. You may notice I said that we define what we want. That was deliberate, since many IaC tools are declarative. The user declares what the outcome should be, and the tool makes decisions about how to get that done
3. Perhaps most importantly, IaC also means treating these files as we would treat any code project. This means thinking like developers; storing and versioning the config assets in a source repository, and applying QA / testing techniques to ensure it does what we intended

Note that these practices should be applied throughout the lifecycle (not just when provisioning new infrastructure) in order to gain the most value from IaC, and avoid configuration drift through manual / console driven change.

Who is it for?

IaC is for any team who wants to make infrastructure delivery more repeatable, reliable and ultimately faster. Organisations will also benefit from infrastructure version control, and a relatively transparent way of collaborating on Infrastructure stacks where multiple team members (or teams) are working on the same environments.

Yes, but who in the organisation will use it?

To adopt IaC, it is vital to consider operating models for the teams involved in infrastructure and their customers - the development teams.

Infrastructure teams will have a steep learning curve ahead in adopting developer practices, and may need support from the developer teams.

Organisations may eventually aim to develop modular infrastructure templates, which include "must have" configurations reflecting company or legislative policy or architecture choices. These could be created by an IaC specialist pool, and reused by other members of the infrastructure team to build systems out of re-usable components. These components are then in turn used by development teams to "self-serve" ; such as when automating the provision of a test environment to execute against (prior to destroying it to save ongoing costs). You may already have product teams with developers working alongside cloud-engineers to facilitate this.

Sidebar: The CI/CD Project Use Case

IaC can be considered a key enabler for CI/CD projects, where most of the focus change is on the automation of code delivery. Organisations may only?attain the agility expected if their underlying?environments, infrastructure and platforms can also be deployed reliably, consistently and?in in a timely manner. Cloud usage alone won't solve this problem without automating the provisioning & configuration processes, and it's easy to end up with expensive sprawl of unused infrastructure wherever administrators don't have time to stand-down environments for fear of the manual effort in standing them up again. In short - do the hard work once, to make it simple multiple times!

How should we approach this?

Assuming you have established support for an IaC project, I'd recommend an initial "vision" phase to create a broad-and-shallow view of how you intend to use IaC at your organisation. Create a high view of how IaC will be applied and by whom, including the impacts on the roles in the organisation, and plan for different phases of adoption.

One you have this vision it may be obvious where to start based on the issues you want to solve most. I usually encourage clients to deliver value incrementally, and IaC projects are no different. There are many places to start, but here are a few suggestions:

• while I've said IaC should be applied across the life-cycle, perhaps your first steps could be limited to the provision of core components (servers, load balancers, storage) with the value delivered being that non-prod can be more agile (admins won't be reluctant to stand-down expensive cloud infrastructure if they know that the core can be re-created easily)
• focussing on the application-components which are harder to configure manually, or which are most often changed (e.g. automating web/app platforms)
• try out IaC in a new project and for new infrastructure requirements, and only later "radiate" the techniques into other systems and teams.

For any adoption of disruptive change, a common mistake is to wait for a project which will deliver the full solution (one client was unwilling to invest in projects which didn't guarantee a "big red button" environment build, while 90% of the benefits were really actually available for 40-50% of the effort).

IaC Tools and Packages

The tooling selected by your organisation will be important. There are many tools on the market, and you will need a small suite of them in all likelihood. Some you may already have, or may be "Native" to your chosen public cloud provider.

Side Note: Before going too much further on this topic, I'd like to say that my experience is that the tool choice debate has an odd habit of dominating the conversation among the teams. I encourage project leads to ensure the focus of the project stays on business value and avoid "tools for tools sake". It should also be remembered also that every tool has an overhead and a cost, not least in training and updates, even if it appears cheap or free to licence.

Be wary of those who would swap tools too readily or without considering the wider impacts.

Tools tend to sit at various layers:

• Tools which provide the IaC provisioning and configuration (e.g. Terraform for the provisioning of an infrastructure stack, or Ansible for configuration management)
• Optionally, orchestration tools to string all of together (Jenkins, GoCD)
• Tools which support IaC by providing facilities to manage the code and/or apply it (e.g. GitHub, Key & Secrets Vaults, Artifact Repositories like jFrog)
• Existing business process tools (e.g. ServiceNow, which may trigger activities based on a service request or other)

You can bet that (just like that DIY project you started at home) you will be "back to B&Q" more than once before you've landed on the right tool set, so a good dose of agile-style experimentation is to be encouraged and should be factored into that initial project activity. It's OK to get things wrong or want a "do-over" when trying new things; persevering with the wrong tools though a feeling of being personally invested will only lead to technical debt and problems down the line.

Wrapping Up

When it comes to disruptive change in IT, sales teams like to focus on the benefits, but tend to underestimate the hard-graft and complexities in making it successful. I suspect anyone who's migrated production systems of any scale and importance into the cloud can attest to this!

However, just like that Cloud migration, the prize for implementing IaC can be significant, and we should not forget the significant limitations of existing, manual / console driven processes.

The good news is that every iteration of work to build up your IaC skills, processes and assets will likely teach you more about the art of the possible, and deliver value in itself, even if you go no further.