Securely Passing Environment Variables - CodeCommit + CodeBuild + CodeDeploy | DevOps With AWS Part 4

There is a genuine problem for developers to decide how to pass secure credentials in applications.

You can always have your credentials set as JSON file or env file but!!! that's dangerous ?? ?? , You can, doesn't always mean you should, and here are the reasons:

1) If you store super-secret credentials in the git repo as JSON or env, there always a chance of a breach, leak, and/or other issues.

2) Anybody, who has the access to repo can see or can change, which probably you don't want.

3) It's not a dynamic approach.

To overcome this issue, there are multiple ways, out of which below way it can be achieved with help of AWS tools/services.

Repo link: https://github.com/sd031/aws_codebuild_codedeploy_nodeJs_demo

Let learn how to pass env variables step by step and (if you would like to see a demo, feel free to watch the above video! ):

Step 1: Add the buildspec.yml file into AWS CodeCommit for a Code build project to integrate:

What is AWS CodeCommit?

AWS CodeCommit is a highly scalable, managed source control service that hosts private Git repositories. You simply create a repository to store your code. There is no hardware to provision and scale or software to install, configure, and operate. CodeCommit helps you collaborate on code with pull requests, branching, and merging. You can implement workflows that include code reviews and feedback by default, and control who can make changes to specific branches. You can use CodeCommit to securely store anything from source code to binaries, and it works seamlessly with your existing Git tools.

So Push your repo into AWS CodeCommit for easy integration with AWS CodeBuild and other AWS DevOps Tools!

What is AWS CodeBuild?

AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy. With CodeBuild, you don’t need to provision, manage, and scale your own build servers. CodeBuild scales continuously and processes multiple builds concurrently, so your builds are not left waiting in a queue. You can get started quickly by using prepackaged build environments, or you can create custom build environments that use your own build tools. With CodeBuild, you are charged by the minute for the compute resources you use.

Here's an example buildspec.yml file:

version: 0.2
phases:
  install:
    runtime-versions:
      nodejs: 12
    commands:
      - echo Getting Started
  pre_build:
    commands:
      - echo Installing some packages and source NPM dependencies...
      - npm install -g json
      - chmod +x ./build_scripts/create_configuration.sh
      - sh ./build_scripts/create_configuration.sh
      - npm install
      - aws --version
  build:
    commands:
      - echo Build started on `date`
      - echo Run all of your build related code if any
      - echo Build finished, now moving to S3
  post_build:
    commands:
      - echo Build completed on `date`
      - aws deploy push --application-name "${CODE_DEPLOY_APPLICATION_NAME}" --s3-location "s3://${CODE_DEPLOY_S3_BUCKET}/codedeploydemo/app.zip" --ignore-hidden-files --region "${AWS_REGION}"

As mentioned in the pre_build stage, the create_configuration.sh file contains the logic which fetching environment variables required for the project and making a JSON file from the environment variables:

Here's an example of create_configuration.sh file:

#!/bin/bash
	CONFIG_PATH="./config/production.json"
	echo Check Below
	echo $CONFIG_PATH
	mkdir config
	echo "{}" >> $CONFIG_PATH
	echo $PWD
	json -I -f $CONFIG_PATH \
	      -e "this.DB_HOST='$DB_HOST'" \
	      -e "this.DB_USERNAME='$DB_USERNAME'" \
	      -e "this.DB_HOST='$DB_HOST'" \
	      -e "this.DB_USERNAME='$DB_USERNAME'" \

Step 2: Create an AWS CodeBuild Project (check the video for steps) and set-up environment variables:

While creating the project in the environment sections there is an option to add environment variables or later you can add the environment variables by clicking on the edit environment option:

Now, the most important part! , in name mention the environment variable name, in value, you can set the value but that value is plain text and not encrypted, so for normal environment variables it's ok to set the actual value in the value section and set type as plaintext, but if you want to secure it securely, store the secure value in AWS parameter store, then set type as Parameter and set the put the key of AWS parameter store as the value, and that's it, now your environment variables are full secure!

Step 3: Create appspec.yml file to use AWS CodeDeploy for Application Deployment:

But make sure your AWS instances have AWS code deploy agent installed, up and running (Follow video if you are not sure how )

What is AWS CodeDeploy:

AWS CodeDeploy is a fully managed deployment service that automates software deployments to a variety of computing services such as Amazon EC2, AWS Fargate, AWS Lambda, and your on-premises servers. AWS CodeDeploy makes it easier for you to rapidly release new features, helps you avoid downtime during application deployment, and handles the complexity of updating your applications. You can use AWS CodeDeploy to automate software deployments, eliminating the need for error-prone manual operations. The service scales to match your deployment needs.

appspec.yml file:

version: 0.0
	os: linux
	files:
	  - source: /
	    destination: /home/ubuntu/app
	hooks:
	  ApplicationStop:
	    - location: deployment_scripts/stop_server.sh
	      timeout: 300
	      runas: root
	      
	  BeforeInstall:
	    - location: deployment_scripts/before_install.sh
	      timeout: 300
	      runas: root
	

	  AfterInstall:
	    - location: deployment_scripts/after_install.sh
	      timeout: 300
	      runas: root
	

	  ApplicationStart:
	    - location: deployment_scripts/start_server.sh
	      timeout: 300
	      runas: root
	

	  ValidateService:
	    - location: deployment_scripts/validate_service.sh
	      timeout: 300

	      
          runas: root

If you would like to learn more about CodeDeploy, Read this article and attached video: https://www.dhirubhai.net/pulse/aws-codedeploy-devops-part-3-sandip-das/

Create CodeDeploy Application, code deployment group.

Step 4: Setting up AWS CodeBuild project related env variables in AWS CodeBuild Project and start the build from AWS CodeBuild

You Might have noticed this line in buildspec.yml file

 - aws deploy push --application-name "${CODE_DEPLOY_APPLICATION_NAME}" --s3-location "s3://${CODE_DEPLOY_S3_BUCKET}/codedeploydemo/app.zip" --ignore-hidden-files --region "${AWS_REGION}

After Done setting up the AWS CodeDeplloy project, then add a few more environment variables in the AWS CodeBuild project as follows:

CODE_DEPLOY_APPLICATION_NAME : set value of the AWS CodeDeploy Project name
CODE_DEPLOY_S3_BUCKET: Create a bucket, enable versioning and mention bucket name
AWS_REGION: The region where you have created the AWS CodeDeploy Project

After you have set the required fields, you can run the build and you will see, after the build finished the build pushed to the s3 bucket and the code is ready to deploy.

Step 5: Deploy the Code:

Go to AWS CodeDeploy Project, then click on created AWS CodeDeploy group, then create the deployment, then in the part where selecting the revision, select the latest available revision, and start the deployment.

After Deployment is done, the latest changes will be deployed to the EC2 instances.

I hope all this knowledge in this article, you are going to apply in your current and future projects and manage code more efficiently ??

References: AWS Official Site Documentation

Also, read (If not already): 

AWS CodeCommit | DevOps With AWS Part 1

AWS CodeBuild | DevOps With AWS Part 2

AWS CodeDeploy | DevOps With AWS Part 3

About the Author:

Sandip Das works as a Sr. Cloud Solutions Architect & DevOps Engineer for multiple tech product companies/start-ups, have AWS DevOps Engineer Professional certification, also holding the title of "AWS Container Hero",

He is always in "keep on learning" mode, enjoys sharing knowledge with others, and currently holds 5 AWS Certifications. Sandip finds blogging as a great way to share knowledge: he writes articles on Linkedin about Cloud, DevOps, Programming, and more. He also creates video tutorials on his YouTube channel.