Everything as Code By Example [Updated: 2022]

(cross-posted from the official klotho?blog post)

It's becoming more popular to manage and operate everything in code, whether it's infrastructure, configuration, security, policies, pipelines, operations, compliance or even documentation. This blog will highlight the different “As Code” solutions with a few examples

Benefits of Everything as Code

There are few recurring benefits that are true for all "As Code" approaches:

Encapsulation and reproducibility of a system. By capturing the necessary elements in code, tools are able to create and recreate a system using a single source of truth.
Version controlled. Versioning makes it easy to track changes and revert back to previous versions.
Collaboration is straightforward. The easier it is for people to understand and contribute to your system, the more likely they are to do so, and we already know how to collaborate on code.

However, a great deal of power comes with a great deal of responsibility. Everything as Code allows teams and companies to control a bunch of systems, which makes it increasingly easy to break one or more of them if guardrails aren't in place. Be careful out there!

The Ultimate Guide

Infrastructure as Code
Configuration as Code
Policy as Code
Security as Code
Pipelines as Code
Monitoring as Code
Docs as Code

Infrastructure as Code

Perhaps the most popular and mature of the lot is Infrastructure as code, or IaC. It’s the process of managing and provisioning on-prem or cloud infrastructure through code.?

For example, deploying a web application and placing a load balancer in front of it and deploy it to AWS. Historically those were popularized by tools like?Terraform?and CloudFormation using text specifications or templates, and more recently through actual code with tools like?Pulumi?and?AWS CDK.

And while It’s easy to overlook on-prem data centers, they still play a large role in the industry, and solutions to automate those?also exist.?For example, a 5-instance hello world application that runs on Fargate in AWS takes 31 lines of code to define:

import * as awsx from "@pulumi/awsx"

// Step 1: Create an ECS Fargate cluster.
const cluster = new awsx.ecs.Cluster("cluster");

// Step 2: Define the Networking for our service.
const alb = new awsx.elasticloadbalancingv2.ApplicationLoadBalancer(
    "net-lb", { external: true, securityGroups: cluster.securityGroups });
const web = alb.createListener("web", { port: 80, external: true });

// Step 3: Build and publish a Docker image to a private ECR registry.
const img = awsx.ecs.Image.fromPath("app-img", "./app");

// Step 4: Create a Fargate service task that can scale out.
const appService = new awsx.ecs.FargateService("app-svc", {
    cluster,
    taskDefinitionArgs: {
        container: {
            image: img,
            cpu: 102 /*10% of 1024*/,
            memory: 50 /*MB*/,
            portMappings: [ web ],
        },
    },
    desiredCount: 5,
});

// Step 5: Export the Internet address for the service.
export const url = web.endpoint.hostname;;

(source:?Pulumi Examples Repo)

Infrastructure as Code scales to larger and more complex systems. For example, you can spin up an entire Kubernetes EKS cluster, with a load balancer using?87 lines of code.

Configuration as Code

Configuration as code is the idea that 1st and 3rd party services you rely on can be configured through code.

For example, when a new employee joins organization, we want to automate the company github in preparation for the on-boarding training event:

// send an invite to the user on the org level
await addUserToOrganization(user, organization)

// create a new repository for this user:
const repoName = `attendee-${user.login}`
await createUserRepo(organization, repoName)

// give the user admin acccess to the repo:
await addUserToRepo(user, organization, repoName)

// add the user to the team for the day of the training:
await addUserToTeam(user, organization, team)

// add the team to the repo (so that the rest of the team can help with PR's):
await addTeamToRepo( organization, repoName, team):

(source)

Policy as Code

(also known as Compliance as Code)

Policy-as-code is all about permissions. What makes them powerful is levels of enforcement, where the organization can set advisory, soft or hard mandatory enforcement levels, and the tools that govern them are able to pass/fail compatible systems from moving forward.

For example, a policy can govern developer access to an organization’s customer data repository. The following policy code shows how to only allow the members of the CMSAdmin group to see customer contracts:

package httpapi.authz

import input

# Allow CMS admin members access to customer contracts.
default allow = false
allow {
  input.method == "GET"
  some contractId
  input.path = ["contracts", contractId]
  input.user == cmsAdmins[_]
}

# Only the following belong to the CMS admins group
cmsAdmins = ["ndrew", "mwang", "fhardy"]

(source)

Security as Code

Security has been a manual process driven by SecOps in most larger organizations, with manual checklists and dev team pushback to implement security features as an afterthought.

With Security as Code, security is integrated into the entire development lifecycle. Issues can be detected using static application security testing (SAST), including finding and automatically fixing vulnerabilities in the code, open source dependencies, containers, and infrastructure as code.

For example, by integrating with Github Actions, Snyk can scan for vulnerabilities on every git push:

name: Example workflow using Snyk
on: push
jobs:
  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@master
      - name: Run Snyk to check for vulnerabilities
        uses: snyk/actions/node@master
        env:
          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
        with:
          command: monitor

(source)

Pipelines as Code

(source)

Now that we have infra, security and policy as code, we need a release process to put it all together for multiple environments. Teams want to configure builds, tests, and deployment and structure larger and interdependent flows.

That’s where tools like Github Actions, CircleCI, TravisCI and Jenkins come in handy. For example, we can add a pipeline action that approves and attempts to merge pull requests based on branch protection rules (e.g. require branches to be up to date, require status checks to pass):

name: Merge me!

on:
  workflow_run:
    types:
      - completed
    workflows:
      # List all required workflow names here.
      - 'Continuous Integration'

jobs:
  merge-me:
    name: Merge me!
    runs-on: ubuntu-latest
    steps:
      - # It is often a desired behavior to merge only when a workflow execution
        # succeeds. This can be changed as needed.
        if: ${{ github.event.workflow_run.conclusion == 'success' }}
        name: Merge me!
        uses: ridedott/merge-me-action@v2
        with:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

(source)

Monitoring as Code

(also known as Operations as Code)

(source)

Comprehensive monitoring as code covers and codifies the collection, diagnosis, alerting, processing, and remediation (self-healing). For example, using Checkly and Terraform, we can define and monitor a login test check on every deploy:

resource "checkly_check" "login" {
 
  name                      = "Login Flow"
  type                      = "BROWSER"
  activated                 = true
  should_fail               = false
  frequency                 = 10
  double_check              = true
  ssl_check                 = false
  use_global_alert_settings = true
  locations = [
    "us-west-1",
    "eu-central-1"
  ]
 
  script = file("${path.module}/scripts/login.js")
 
}

(source)

Docs as Code

Documentation is as essential as the developer platform you’re building, whether internal or external. This is a growing practice of treating documentation development using the same code tools and techniques like version control (GitHub) and automation (CICD) with static site generators (SSG) and more.

Documentation can be anything from tutorials to auto-generated reference API docs. For example, when a new markdown documentation page is modified or created, check out the code, build the static version of the documentation page, then upload the files to the public web site using SFTP:

on: [push]

jobs:
  deploy_job:
    runs-on: ubuntu-latest
    name: deploy
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: deploy file
        uses: wlixcc/[email protected]
        with:
          username: 'root'
          server: 'your server ip'
          ssh_private_key: ${{ secrets.SSH_PRIVATE_KEY }} 
          local_path: './static/*'
          remote_path: '/var/www/app'
          args: '-o ConnectTimeout=5'

(source)