Day 27 : Infrastructure as Code (IaC) #90DaysofDevOps

What is infrastructure as code?

Infrastructure as code, also referred to as IaC, is an IT practice that codifies and manages underlying IT infrastructure as software. The purpose of infrastructure as code is to enable developers or operations teams to automatically manage, monitor and provision resources, rather than manually configure discrete hardware devices, operating systems (OSes), applications and services. IaC is sometimes referred to as programmable or software-defined infrastructure.

The concept of IaC is similar to programming scripts, which are used to automate IT processes. However, scripts are primarily used to automate a series of static steps that are repeated numerous times across multiple servers. IaC uses higher level or descriptive language to code more versatile and adaptive provisioning and deployment processes. For example, IaC capabilities included with Ansible, an IT management and configuration tool, can install MySQL server, verify that MySQL is running properly, create a user account and password, set up a new database and remove unneeded databases.

The code-based infrastructure automation process closely resembles software design practices in which application development teams carefully develop declarative code, control code versions, test iterations and limit deployment until the software is approved for production. IaC is also frequently associated with provisioning and managing cloud infrastructure, using a cloud provider's application programming interfaces (APIs) to access and interact with the provider's resources and services.

How does infrastructure as code work?

In simplest terms, IaC uses software code to provision and manage an IT infrastructure. By using code to define a desired IT outcome, organizations can improve infrastructure consistency, security, automation and performance.

A lot of time and effort is involved in manually provisioning and configuring servers, storage, networks, OSes and applications, as well as other devices and services, such as databases, load balancers and firewalls. It can take hours -- even days -- for a human to provide a suitable deployment environment for an enterprise workload and its data. Human error can creep into many aspects of this manual effort, leading to the following problems:

• Inconsistent outcomes where no two environments are the same.
• Security vulnerabilities when settings and configurations are done incorrectly.
• Troubleshooting difficulties where infrastructure documentation is poor or nonexistent.
• Compliance and business continuance issues caused by security and troubleshooting problems.
• Problematic management and change control.

IaC embodies all the instructions needed to build and deploy a desired IT environment as a software entity. By creating a detailed set of instructions, developers and IT staff can see the steps required and measure the outcomes of the code.

The infrastructure code delivers reproducible results and can be executed at the push of a button or in response to specific conditions. It brings a high level of automation to provisioning and management.

Change is easier to manage, because any changes to the infrastructure code results in a new version that is well-documented and managed with any common version control system. If a problem arises as the result of a change, it can be corrected simply by running a previously known good version of the code. This simplifies troubleshooting and improves compliance and security for the business.

The following four basic types of IaC are often used in tandem:

• Scripts are often used to automate simple ad hoc tasks.
• Configuration management code defines and automates the installation and configuration of devices such as servers.
• Provisioning code automates the provisioning of an overall infrastructure, tying varied aspects of the environment together.
• Container code relies on templates to define image files with libraries and dependencies needed to deploy and execute an application.

The IaC process typically involves these three steps:

1. A developer creates specifications for the desired infrastructure using a domain-specific language.
2. The specification files are sent, or staged, to a server or repository such as GitHub or an API.
3. The IaC platform uses the specification files, either automatically or on-demand, to create and configure the intended infrastructure.

IaC approaches: Declarative vs. imperative

Infrastructure-as-code tools operate using declarative and imperative approaches.

Declarative approach

A declarative programming approach outlines the desired, intended state of the infrastructure, but doesn't explicitly list the steps to reach that state. In effect, a declarative approach states an intended goal and then allows the underlying IaC tools to decide how those goals are implemented. SQL is a commonly known declarative programming language. Amazon Web Services (AWS) CloudFormation templates, among others, are written in the declarative style of infrastructure as code.

Imperative approach

An imperative programming approach defines commands that enable the infrastructure to reach the desired state. It defines the precise steps and their order to achieve a desired outcome and doesn't deviate from those steps. Object-oriented languages, such as C++ and Java, can be used for imperative programming. A tool such as Chef can be used in the declarative manner, but also imperatively as needed.

In both approaches, IaC is configured on a template, where the user specifies the resources needed for each server in the infrastructure. The template is used to either verify that a system is properly configured or to put it in the appropriate setup. Templates can be constructed as a set of layers of resources, such as in AWS CloudFormation, which makes a stack.

IaC best practices

Infrastructure as code involves infrastructure configuration and infrastructure management with high levels of automation. While there is no single universal approach to deploying and using IaC, there are numerous best practices that help streamline these deployments. Common IaC best practices include the following:

• Use version control. Provisioning and configuring infrastructure through software means that IaC adheres to many software development best practices such as strong version control. Organizations should use repositories for IaC files, comprehensive version control, and strong documentation and recordkeeping to track who accesses the files and what changes occurred between versions.
• Focus on immutable infrastructure. Software defines an infrastructure. With IaC, an immutable infrastructure approach should be used. Infrastructure changes shouldn't be made once the code is provisioned and deployed. If a change is needed, the current infrastructure should be deleted, and a new infrastructure, using new versions of the IaC files, should be deployed. This prevents any chance of configuration drift.
• Don't share secrets. Many configuration elements involved in IaC require passwords or encryption certificates. These are often included in IaC files without protection, potentially exposing sensitive systems and compromising security. Use a separate secrets manager to store identity and access management details in a protected manner.
• Keep files secure. IaC is intended to embrace the same collaborative attitude as other DevOps and Agile software design. However, there's likely to be some intellectual property, secrets or other sensitive content in IaC files. It's important to keep them as secure as other source code used by the business.
• Test and validate files. IaC files should never be taken for granted. They should be tested and validated in the same ways that other software is tested before being released to production. This often involves using continuous integration/continuous delivery (CI/CD), DevOps and Agile techniques.
• Use smaller files. Use many smaller IaC files to perform specific tasks and then connect those files with a script or another IaC template. Using a larger number of smaller files allows better modularity and facilitates reuse of IaC files, rather than creating and maintaining a single large ubiquitous file to do everything.
• Monitor and correct drift. When implemented properly with careful attention to immutability, IaC ideally shouldn't allow configuration drift. However, drift can still occur due to system problems, troubleshooting and human error. It's important to monitor the infrastructure environment, compare the environment against the IaC definition to find instances of drift, and document and correct any instances of drift.

Benefits of infrastructure as code

There are many benefits associated with IaC, from efficiency of automation to its flexibility to align with other modern IT practices. IaC benefits include the following:

• Speed and efficiency. Automated provisioning and management are faster and more efficient than manual processes. This not only extends to provisioned resources and virtualization, but also to databases, networking, user account management and other tied-in services. IaC also can include code that automatically scales by adding or shutting down environments and resources when they're no longer needed.
• Consistency. Software developers can use code to provision and deploy servers and applications according to business practices and policies, rather than rely on systems administrators in a DevOps environment. A developer might create a configuration file to provision and deploy a new application for quality assurance or experimental deployment before operations takes over for live deployment in production.
• Accountability. Code and corresponding repository management provide a solid foundation for documenting IaC code. It's easy to see factors such as who developed each version and what changes occurred between IaC code versions. This strengthens business compliance and code quality standards.
• Cost. Implementing IaC requires a considerable investment, but the payoff in speed, efficiency, consistency, compliance, troubleshooting and automation are typically worth the investment. IaC replaces a manual process that can take hours with an automated process that can be measured in minutes. It eliminates many problems associated with manual infrastructure provisioning and configuration.
• Alignment with DevOps. With the infrastructure setup written as code, it can go through the same version control, automated testing and other steps of a CI/CD pipeline that developers use for application code. An organization might combine IaC with containers, which abstract the application from the infrastructure at the OS level. Because the OS and hardware infrastructure are provisioned automatically and the application is encapsulated atop it, these technologies prove complementary for diverse deployment targets, such as test, staging and production.

Challenges of infrastructure as code

Despite its benefits, IaC poses the following potential disadvantages:

• Tools. Infrastructure as code requires additional IaC-specific or -capable tools, such as configuration management tools and an automation/orchestration system that could introduce learning curves and errors. Mistakes can proliferate quickly through servers, especially where there's extensive automation, so it's essential to monitor version control and perform comprehensive prerelease testing.
• Feature sets. IaC tools provide an assortment of powerful features and capabilities, but some features might be incomplete or missing all together. It's important to evaluate and vet tools, especially against business needs, before making an IaC investment and maintain careful oversight of each tool providers' product or feature roadmap.
• Complexity. The introduction of new tools, processes and standards into the DevOps and operations environments can add complexity to an area already rife with varied tools and platforms. Enterprise-wide support is required for IaC to provide lasting and meaningful benefits to the business.
• Change. If administrators change server configurations outside of the set IaC template, there's potential for configuration drift without the use of additional change management tools. It's important to fully integrate IaC into systems administration, IT operations and DevOps practices with well-documented policies and procedures. If legacy security and monitoring tools aren't up to handling IaC, this will require additional investments in more tools, with additional training and testing to integrate them into workflows.
• Skill sets. Another challenge with IaC is that it places more responsibility on developers to understand how to write efficient code that translates seamlessly into production environments. They must also have strong knowledge of languages used for IaC however an organization has implemented it, such as JSON, YAML, Ruby, C++ and SQL. Knowledgeable staff might be harder to find and retain.

Infrastructure as code and cloud computing

Cloud computing shares a general starting vision with IaC: IT resources such as compute, storage and networking are abstracted from physical hardware, tied to additional services and loaded into instances that are spun up and down as needed through the cloud provider's APIs.

IaC takes this a step further, to automate this process using predefined sets of instructions to do the following:

• Provision resources.
• Configure the instance.
• Configure and deploy a workload into the instance.
• Connect associated services.
• Monitor and manage the deployment over time.

This depth of automation is especially important because of the cloud's vast array of applications, services and functions, which are stacked together and mainly connected through APIs. The scale and scope of cloud requires an automated governed process, rather than doing everything manually. Organizations with hybrid cloud environments benefit even more because templated configurations and resources can be applied across multiple environments.

Immutable vs. mutable infrastructure

Mutable infrastructure refers to the practice whereby infrastructure components are changed in production, while the overall service or application continues to operate as normal. Immutable infrastructure assembles and sets components and resources to create a full service or app. If a change is required for any component, they aren't changed or reconfigured, they're all updated and effectively redeployed in an instance. A new iteration is assembled, tested, validated and launched, while the old iteration is discontinued and its resources released for reuse.

Immutable infrastructure has gained favor particularly for cloud and microservices environments, which are highly scalable and involve many interdependent components and services. Any one-off updates to address specific issues can cause configuration drift that cascades as updates are rapidly pushed to production. It's more efficient to reissue sets of immutable services and components than to patch and reconfigure individual infrastructure components. IaC emphasizes the use of immutable infrastructure techniques, though immutability isn't a prerequisite for IaC.

Infrastructure-as-code tools

IaC tools configure and automate the provisioning of infrastructure. These automation tools can automatically execute the deployment of infrastructure, such as servers, with orchestration functionality. They also can configure and monitor previously provisioned systems.

IaC tools enforce the setup from the template via push or pull methods. In the push method, a centralized server sends the desired configuration to a specific system or systems. The pull method is initiated by a request to a centralized server from a system or systems in the infrastructure. Tools typically are designed by default for push or pull deployment of code, but they can be set up for specific instances to do the other. These tools should also be able to roll back changes to the code, as in the event of unexpected problems from an update.

Examples of infrastructure-as-code tools include AWS CloudFormation, Chef, Google Cloud Deployment Manager, HashiCorp Terraform, Microsoft Azure Resource Manager, Puppet, Red Hat Ansible and SaltStack. Some tools rely on a domain-specific language, while others use a standard template format, such as YAML and JSON.

When selecting a tool, organizations should consider the target deployment and select a tool designed for that environment. For example, AWS CloudFormation is designed to provision and manage infrastructure on AWS, and it works with other AWS offerings. Similarly, Azure Resource Manager is the tool used to manage infrastructure on the Azure platform, while Google Cloud Deployment Manager is Google's infrastructure deployment service. Alternatively, Chef works with on-premises servers and multiple cloud provider infrastructure-as-a-service offerings.