Using GitHub self-hosted runners in AWS (part 1)

GitHub provides a very quick and easy way to get started running CI/CD flows through GitHub Actions, which use their GitHub runners as build and deploy agents. There are some ways to configure these runners, and they are charged in an on-demand capacity, making them a very flexible option to get started on projects.

But what if you would like to have a similar flexibility while being able to ensure that your CI/CD runners are secure? From an Info-Sec perspective, letting third parties have the possibility of having build+deploy capabilities on your production accounts might not be desirable.

Luckily, GitHub offers another option for these situations: self-hosted runners. By installing a package in an instance (be it a full server, VM, or a container), you can register the agent as a fully-fledged GitHub runner in your repository, project or organization.

There's some amazing stuff you can do with this registration, like tagging certain runners to be available to specific projects or workflows; for example, those requiring a specific OS, binaries or computing power.

For these articles I'm only going to attempt to show an interesting way of setting up GitHub runners in AWS in an on-demand capacity. That is, runners will never be "idle", and they should only exist when a workflow needs a runner. Any additional logic with regards to handling more complex scenarios with multiple types of runners will be left as an exercise to the reader.

The main motivation is to show how to make use of self-hosted runners while also keeping costs down and still having a nimble setup that can be customized further. Another requirement is that these runners should be able to build containers, which is a rather limited option in AWS as far as "on demand" building goes due to privileges requirements.

To begin with, we'll be running these self-hosted runners as ephemeral builders. Our Dockerfile will include all dependencies to be able to fully install and run the runner package, but as of now not much else:

FROM debian:stable-slim
USER root


ARG TOKEN="YOUR_TOKEN_HERE"
ARG GH_URL="YOUR_GH_URL_HERE"
ARG USER="runner"
ARG GROUP="${USER}"
ENV envTOKEN=${TOKEN}
ENV envGHURL=${GH_URL}


RUN apt-get update && apt-get upgrade -y && \
? ? apt-get install curl gpg -y && \
? ? mkdir gh_runner && cd gh_runner && \
? ? curl -o actions-runner-linux-x64-2.299.1.tar.gz -L https://github.com/actions/runner/releases/download/v2.299.1/actions-runner-linux-x64-2.299.1.tar.gz && \
? ? echo "147c14700c6cb997421b9a239c012197f11ea9854cd901ee88ead6fe73a72c74 ?actions-runner-linux-x64-2.299.1.tar.gz" | sha256sum -c && \
? ? tar xzf ./actions-runner-linux-x64-2.299.1.tar.gz && \
? ? ./bin/installdependencies.sh && \
? ? rm -f ./actions-runner-linux-x64-2.299.1.tar.gz && \
? ? groupadd ${GROUP} && useradd ${USER} -g ${GROUP} && \
? ? chown -R ${USER}:${GROUP} /gh_runner


USER ${USER}
WORKDIR /gh_runner


CMD ./config.sh --url ${envGHURL} --token ${envTOKEN} --ephemeral && ./run.sh        

There is still a bit of configuration work left to be done here, but that will be for the future. For now, we need to understand that the TOKEN argument we need to feed this runner is given to us by GitHub as a 1-hour token that can register any number of runners.

If you want to test building this Dockerfile, you can try with:

docker build ./ --build-arg TOKEN=mytoken --build-arg GH_URL=github.com        

It's still missing the arguments that will allow it to register itself as a builder in our project. Next time we'll look at how to properly get them from GitHub and, even better, fetching them at runtime.