DevOps on GitLab for Google Cloud Platform (Part 2/2)

We wish to use SSH authentication in the GitLab CI/CD pipeline for Linux VMs on GCP. The pipeline should push the code changes to the production VM and get deployed.

SSH Authentication on GitLab

Before we get started on VMs on GCP, let's refresh how we set up git in a local system, create a new git project locally and push it to the Git server using SSH. We will use GitLab as a remote git server in this section. You'll need to install Git Bash for using git on windows then create an SSH key pair.

Add the public key to your GitLab profile.

Finally, test if the setup was successful with 'ssh -T' command. You should get a welcome message like this.

Now it's business as usual. You can use the git commands to commit and push codes to the remote repository. We will refer to this section in later sections of this article.

SSH Authentication for VMs on GCP

You can create an SSH key pair as we did for GitLab and add the public key to the VM when creating it. You can add more SSH keys later as well if you need to.

Now you can log in to VM by inserting the private key in the Putty. Read more at OS Login-managed SSH connections.

Auto-deployment demo using Git hooks

So far we completed SSH authentication using git. Let's set up a quick Python Flask app in development mode for trying a simple deployment scenario with Git. A bare git repository can be used as a remote repository for pushing codes. The githooks can be used for running deployment scripts after the code is pushed to production.

Remote server setup

We need to initialize a bare git repository that will not contain any codebase but retains the revisions of code.

git init --bare myapp.git

Then we need to create a git hook called post-receive in the hooks folder. It executes on the remote repository once after all the refs have been updated.

cd myapp.git/hooks

nano post-receive

This bash file below checkouts the files it received from the push to the working git repository. You can then cd into the working directory folder to install the dependencies, run build scripts, and finally reload the app.

#!/bin/sh


# Checkout files
git --work-tree=/home/manoj.sahukar/myapp --git-dir=/home/manoj.sahukar/myapp.git checkout -f

# Install dependencies
cd /home/manoj.sahukar/myapp
pip install requirements.txt

# Restart app
# pm2 reload myapp

Then use chmod +x on your script. It means, that you'll make it executable. Right-click on your script and choosing Properties -> Permissions -> Allow executing file as a program gives you exactly the same result as the command on the terminal.

chmod +x myapp.git/hooks/post-receive

Local Repository Setup

Initiate a working git repository with hello.py and start.sh below.

hello.py

from flask import Flask

app = Flask(__name__)
#app.run(debug=True)

@app.route("/")
def hello_world():
    return "<p>Hello, there!</p>"

You can add requirements.txt for installing the required dependency before reloading the app in production.

start.sh

export FLASK_APP=hello
export FLASK_ENV=development
flask run --host=0.0.0.0 --port=8080

Setting FLASK_ENV=development to ensure the app is reloaded when files are updated. Don't use this in a production environment. My goal is here only to demonstrate git deployment.

Now add remote production repository as follows and commit your files.

git remote add live [email protected]:myapp.git

Then push the code to production.

git push -u live master

Now start the server by running the start.sh script.

Make changes to the python file in the local repo and push it to production. You'll notice that the post-receive hook checked out the changes to the working repository and the change is detected by the Flash development server.

Auto-deployment setup on GitLab for VMs on GCP

Finally, we have to combine all the above learning to log in to VM from GitLab and push codes using the SSH protocol however with GitLab CI/CD Pipeline. Although GitLab currently doesn’t have built-in support for managing SSH keys in a build environment they have provided great documentation to work around this limitation and an example project for getting started.

Let's have a look at the yml file from the example project which is modified to include the git deployment commands we used from the local system.

image: ubuntu

before_script:
  ##
  ## Install ssh-agent if not already installed, it is required by Docker.
  ## (change apt-get to yum if you use an RPM-based image)
  ##
  - 'command -v ssh-agent >/dev/null || ( apt-get update -y && apt-get install openssh-client -y  && apt-get install git -y)'

  ##
  ## Run ssh-agent (inside the build environment)
  ##
  - eval $(ssh-agent -s)

  ##
  ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
  ## We're using tr to fix line endings which makes ed25519 keys work
  ## without extra base64 encoding.
  ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
  ##
  - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -

  ##
  ## Create the SSH directory and give it the right permissions
  ##
  - mkdir -p ~/.ssh
  - chmod 700 ~/.ssh

  ##
  ## Assuming you created the SSH_KNOWN_HOSTS variable, uncomment the
  ## following two lines.
  ##
  - echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
  - chmod 644 ~/.ssh/known_hosts


  ##
  ## Optionally, if you will be using any Git commands, set the user name and
  ## and email.
  ##
  - git config --global user.email "[email protected]"
  - git config --global user.name "manojsahukar"

Deploy through SSH:
  script:
  - echo "Added remote git url and pushing it to production"
  ## Add remote server address
  - git remote add live "$REMOTE_URL"
  # - git status
  ## Push the master branch
  - git push live HEAD:master
  - echo "Pushed to production"
  only:
  - master

The YML script uses the following CI/CD variables.

SSH_PRIVATE_KEY

We have already discussed how to create SSH key pairs in the 1st part of this series using the following command. Copy the private key to this variable. id_rsa is the file that contains the OpenSSH private key under c:\users\username\.ssh.

ssh-keygen -t rsa -C "[email protected]" -b 4096

REMOTE_URL

This variable is the complete ssh address of the remote bare repository in VM. As per the deployment project shown before the variable would be as follows.

ssh://[email protected]:myapp.git

SSH_KNOWN_HOSTS

This variable can be found by using the ssh-keyscan utility. '-T' stands for timeout if there is any network delay. Save the output of the following command to this variable.

ssh-keyscan -T 10 xx.xx.xxx.xxx

That's it. Now we can easily commit changes on the GitLab repository and the changes will reflect on the live server automatically.

We are currently implementing a staging server as our IoT Platform passed the initial viability test with a few active users at IIT Delhi. In the upcoming articles, I'll cover the staging strategies for VMs on GCP we are planning to use which is important for integration testing before committing to a live server.

Comments are appreciated. Thanks for making it through the article.

Read more:

• Difference between a Bare and non-Bare repository?
• What does “chmod +x <filename>” do ?
• githooks - Hooks used by Git
• git-checkout