Administering Terraform Cloud using GitHub Actions
Mattias Fjellstr?m
??♂? @ Accelerate at Iver | Author | HashiCorp Ambassador | HashiCorp User Group Leader
Note that this post is cross-posted here, at my blog mattias.engineer, and at dev.to so you can read it where you like to read things like this the most!
Terraform Cloud is a platform for running Terraform remotely in HashiCorp’s own cloud environment. It simplifies working with Terraform in teams and organizations. Terraform Cloud stores your state files for you. You can connect your Terraform Cloud?workspace?to you Git repository and have Terraform Cloud automatically apply any updates that are committed to your repository.
When you set up a new workspace in Terraform Cloud you can select a repository, a directory in your repository where your Terraform configuration resides, as well as which git branch the workspace should create resources from. A workspace can only be connected to a single git repository and branch. If you want to create several instances of your infrastructure you have to create several workspaces in Terraform Cloud. This is all fine, but is there a way to automate creating new ephemeral workspaces for development branches that you only keep alive for a short while? For this you have to use the Terraform Cloud API. There is no CLI available, which is a bit surprising because HashiCorp has an abundance of different CLI tools for their various products. HashiCorp has kindly developed a Go client for Terraform Cloud/Enterprise, so we are not forced to do all the work ourselves.
Let us use the Go client to develop something useful! One thing that I have searched in vain for is a way to work with Terraform Cloud using GitHub Actions. I have a dream of being able to automatically create Terraform Cloud workspaces for development branches that are also automatically deleted when I merge my changes to the main branch. So in this post I will walk through how I did just that!
This is not a post introducing Terraform Cloud. If you want to use the different GitHub actions I present in this post you will need to have a working Terraform Cloud organization and be familiar with how to create a token for authenticating to Terraform Cloud. If you want to learn more and get started I recommend you go to?developer.hashicorp.com/terraform/cloud-docs
Writing custom actions in Go
I recently got certified in GitHub Actions?and as part of that I learned about creating my own custom actions. I used these skills to create a number of actions that perform small tasks in Terraform Cloud. All my actions can be found here:
In this post I will go through the details of how I created the action for creating new workspaces.
First of all, if you want to create a custom action there are three different options to choose from:
Since HashiCorp has a Go client for Terraform Cloud the natural choice for me was to write my action code in Go, thus selecting the?Docker action?type of action.
Metadata for an action
The metadata for a custom action specifies things like its name, a description, what inputs it takes, what outputs it produces, and properties about what the action consists of. This data is configured in a file called?action.yaml?(or?action.yml). For my custom action the?action.yaml?file looks like this:
name: Create Terraform Cloud workspace
author: Mattias Fjellstr?m (mattias.fjellstrom [at] gmail.com)
description: Create a new workspace in Terraform Cloud
inputs:
organization:
description: Organization name
project:
description: Project name
workspace:
description: Desired workspace name
repository:
description: GitHub repository name
default: ${{ github.repository }}
branch:
description: Git branch name to trigger runs from
directory:
description: Repository directory name containing Terraform configuration
runs:
using: docker
image: Dockerfile
args:
- -organization
- ${{ inputs.organization }}
- -project
- ${{ inputs.project }}
- -workspace
- ${{ inputs.workspace }}
- -repository
- ${{ inputs.repository }}
- -working_directory
- ${{ inputs.directory }}
- -branch
- ${{ inputs.branch }}
The?name,?description, and?author?parts are self-explanatory. The?inputs?section specifies six parameters:
The last section is?runs. This is where I configure that this action is?using: docker?and I provide a( path to my Dockerfile (which incidentally is just?Dockerfile), and I provide my inputs as arguments to my Docker image.
Dockerizing my action
The?Dockerfile?for my action is for a simple Go application without any fancy features:
FROM golang:1.20.2-alpine
WORKDIR /app
COPY ./ ./
RUN go build -o /bin/app main.go
ENTRYPOINT ["app"]
The Go code
The meat of the action itself is located in?main.go. I will go through the relevant parts of the code piece by piece. To parse the input arguments I use the?flag?package:
领英推荐
var organizationName string
var projectName string
var workspaceName string
var repositoryName string
var workingDirectory string
var branchName string
func init() {
flag.StringVar(&organizationName, "organization", "", "Organization name")
flag.StringVar(&projectName, "project", "", "Project name")
flag.StringVar(&workspaceName, "workspace", "", "Desired workspace name")
flag.StringVar(&repositoryName, "repository", "", "Git repository")
flag.StringVar(&workingDirectory, "working_directory", "", "Directory containing Terraform configuration")
flag.StringVar(&branchName, "branch", "", "Git branch name")
}
func main() {
flag.Parse()
// ...
}
I define the various flags in an?init()?function that runs before my?main()?function. The first step of the?main()?function is to parse the flags to obtain whatever values were provided (if any). After that I go through each flag to check if a value was provided, and if not I fall back to using environment variables if they are defined. An example for the organization name looks like this:
if organizationName == "" {
log.Println("No organization name provided, will fall back to environment variable")
_, ok := os.LookupEnv(ENV_TERRAFORM_CLOUD_ORGANIZATION)
if !ok {
log.Fatal("Organization name must be provided as input or as environment variable")
}
organizationName = os.Getenv(ENV_TERRAFORM_CLOUD_ORGANIZATION)
log.Println("Organization name read from environment variable")
}
The actual name of the environment variable is stored as a?const?named?ENV_TERRAFORM_CLOUD_ORGANIZATION. To read environment variables I use the?os?package.
Once the input has been parsed I look for the Terraform Cloud API token that I require to be set as an environment variable, and if I find it I go on to initialize the Terraform Cloud Go client:
token, ok := os.LookupEnv(ENV_TERRAFORM_CLOUD_TOKEN)
if !ok || token == "" {
log.Fatalf("%s environment variable must be set with a valid token", ENV_TERRAFORM_CLOUD_TOKEN)
}
config := &tfe.Config{
Token: token,
RetryServerErrors: true,
}
client, err := tfe.NewClient(config)
if err != nil {
log.Fatal(err)
}
I have imported the Go client as?import tfe "github.com/hashicorp/go-tfe"?and aliased it to?tfe.
Next there are two parts where I search for various things:
With all of that out of the way the last part of the code creates the workspace:
_, err = client.Workspaces.Create(ctx, organizationName, tfe.WorkspaceCreateOptions{
Type: "workspaces",
Name: tfe.String(workspaceName),
AutoApply: tfe.Bool(true),
WorkingDirectory: tfe.String(workingDirectory),
VCSRepo: &tfe.VCSRepoOptions{
Branch: tfe.String(branchName),
Identifier: tfe.String(repositoryName),
GHAInstallationID: gitHubApplication.ID,
},
Project: project,
})
if err != nil {
log.Fatal(err)
}
I configure the workspace using the input arguments and the?project?and?gitHubApplication?that I searched for and hopefully found. I set?AutoApply?to?true?because I want the infrastructure to be created automatically without manual approvals.
That was a short walkthrough of the code, the full source is available at?GitHub.
Writing a GitHub workflow that creates and deletes Terraform Cloud workspaces
With all of my custom actions written and published in their own GitHub repositories I can go on and actually use them for something.
I had an idea that I wanted to create a GitHub workflow where if I create a new pull-request a new Terraform Cloud workspace is created automatically, and an initial run is triggered. Then when the pull-request is closed the workspace and corresponding infrastructure should be removed.
A workflow that does exactly this might look like the following:
name: Sample Terraform Cloud administration for pull requests
on:
# trigger when pull requests are opened or closed
pull_request:
types:
- opened
- closed
# set some convenience environment variables
env:
ORGANIZATION: my-terraform-cloud-organization
PROJECT: my-terraform-cloud-project
jobs:
# job for creating a workspace for new pull requests
create-workspace:
if: ${{ github.event.action == 'opened' }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
# set up environment variables for terraform cloud
- uses: mattias-fjellstrom/tfc-setup@v1
with:
token: ${{ secrets.TERRAFORM_CLOUD_TOKEN }}
organization: ${{ env.ORGANIZATION }}
project: ${{ env.PROJECT }}
workspace: my-application-${{ github.head_ref }}
# create the workspace (the action I went through above!)
- uses: mattias-fjellstrom/tfc-create-workspace@v1
with:
directory: terraform
branch: ${{ github.head_ref }}
# apply a variable set with azure credentials to the workspace
- uses: mattias-fjellstrom/tfc-apply-variable-set@v1
with:
variable_set: azure-credentials
# trigger an initial run
- uses: mattias-fjellstrom/tfc-start-run@v1
# job for deleting a workspace for closed pull requests
delete-workspace:
if: ${{ github.event.action == 'closed' }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
# set up environment variables for terraform cloud
- uses: mattias-fjellstrom/tfc-setup@v1
with:
token: ${{ secrets.TERRAFORM_CLOUD_TOKEN }}
organization: ${{ env.ORGANIZATION }}
project: ${{ env.PROJECT }}
workspace: my-application-${{ github.head_ref }}
# delete infrastructure and terraform cloud workspace
- uses: mattias-fjellstrom/tfc-delete-workspace@v1
I have added a few comments to the workflow to explain what the different actions are doing. A few details I wish to highlight:
All in all I am happy with the state of theses actions right now. If I need additional feature I might add them later on. However, most of all I hope HashiCorp will do one (or both) of the following things:
We will see what the future brings!
??♂? @ Accelerate at Iver | Author | HashiCorp Ambassador | HashiCorp User Group Leader
2 年Ping HashiCorp! What do you think about developing an official CLI for Terraform Cloud and maybe some official GitHub Actions for Terraform Cloud (and maybe even more generally for HashiCorp Cloud Platform)? ??