Serverless CI/CD on the AWS Cloud

This post was originally published here on Caylent.com.

CI/CD pipelines have long played a major role in speeding up the development and deployment of cloud-native apps. Cloud services like AWS lend themselves to more agile deployment through the services they offer as well as approaches such as Infrastructure as Code. There is no shortage of tools to help you manage your CI/CD pipeline as well.

While the majority of development teams have streamlined their pipelines to take full advantage of cloud-native features, there is still so much that can be done to refine CI/CD even further. The entire pipeline can now be built as code and managed either via Git as a single source of truth or by using visual tools to help guide the process.

The entire process can be fully automated. Even better, it can be made serverless, which allows the CI/CD pipeline to operate with immense efficiency. Git branches can even be utilized as a base for multiple pipelines. Thanks to the three tools from Amazon; AWS CodeCommit, AWS CodeBuild, and AWS CodeDeploy, serverless CI/CD on the AWS cloud is now easy to set up.

Understanding the Key Components

To set up a serverless CI/CD pipeline on AWS, there are several key services that you need to use. Naturally, we start with Identity and Access Management (AWS IAM) for managing users, roles, and groups. AWS IAM gives you granular control and allows you to maintain security across the cloud environment.

AWS CodePipeline is the next tool to use. It is Amazon’s own continuous delivery pipeline management tool that is designed to work seamlessly with other AWS services. Using CodePipeline, you can automate a lot of steps that follow a pull request on Git. In fact, CodePipeline can be configured to build, test, and deploy new codes automatically.

As mentioned before, multiple pipelines can be created with this approach. By going serverless, you can separate your staging and deployment environments without having to manually migrate codes and updates between the two. Since automation is the key here, you can also use multiple Git branches to automate tasks such as load testing and A/B testing.

AWS CodeCommit lets you create a repo directly on AWS, but you can use other Git tools for the job. The one big advantage offered by CodeCommit is its seamless integration with other AWS services. If you are using the GUI to configure and manage your CI/CD pipeline, going Amazon-native is definitely the way to go for maximum convenience.

AWS CodeBuild and CodeDeploy further automate your pipeline and allows for the entire CI/CD workflow to remain serverless. CodeDeploy is fully managed with centralized control accessible via AWS CLI or through the AWS Management Console. You also get support for Lambda functions and other services like Fargate.

That actually brings us to the rest of the components. You still use S3 buckets for storage and EC2 instances for computation, but the provisioning of resources is fully automated. A serverless CI/CD pipeline on AWS cloud allows you to stop worrying about additional (often unnecessary) costs while trying to improve speed, agility, and reliability of your development and deployment processes.

Why a Serverless CI/CD Pipeline?

Going serverless offers some key advantages that cannot be achieved otherwise, starting with the aforementioned lower overhead cost. With AWS improving its pay-per-use pricing structure, there is no doubt that you can save a lot on staging and deployment environments alone. There is also the possibility of improving your CI/CD pipeline for better cost-effectiveness.

Improved security is another major benefit you can expect from going serverless. IAM will take care of user, role, and group management using code-based configurations, and you can code how user access is granted as part of your pipeline. Other tools such as Key Management Service and AWS Secrets Manager lets you take security to a whole new level.

There is also a lower chance of errors and bias. With Git as a single source of truth, every code push can be traced back to its origin. You also benefit from Git’s native rollback features and you can pull the plug on a problematic update without having to bring down the entire application. Everything is modular and compartmentalized.

Going Serverless

There are pipeline templates that you can use to get started, including one designed for enterprise users of AWS. You also have the option to start from scratch but starting with a template lets you skip the usual complications of setting up the cloud environment for rapid CI/CD workflows.

You start by creating a sub-account for sharing services using AWS IAM and then assigning the least required permissions to that account. Enabling cross-account access for development and deployment can also be done through IAM.

At this point, you have the option to separate the accounts used for development and deployment, with the former mainly used for staging and testing purposes. Separating the two lets you fortify your production environment fully.

The AWS CloudFormation template is a good starting point if you want to rapidly deploy resources. Make sure you choose a region that supports the tools used by this template, and you are all set in just a few minutes.

The real challenge is configuring the pipeline, but this too is made manageable with tools like AWS SAM and AWS Lambda. Install Git and ssh-keygen, and then connect to your Git repository to start building your pipeline. Test a code push with AWS CodeCommit.

Once you have reached this point, you can customize the rest of the pipeline from CodeCommit. The visual tools provided by this service makes assigning deployment templates, configuring parameters, and even managing secrets very easy to do.

As an added bonus, you can automate testing and code review by integrating AWS CodeBuild. Simply define your code style by writing a CodeBuild build specification file and you will have automated testing added to your CI/CD pipeline.

All of these services run without taking up valuable cloud resources or requiring permanent allocation. Once set up, your serverless CI/CD pipeline is now completely serverless, and all that’s needed to update your apps and services is a Git commit.

For more on optimizing your AWS environments with the right services, read Cloud Services Comparison: AWS vs. Google vs. Azure.

Caylent provides a critical DevOps-as-a-Service function to high growth companies looking for expert support with Kubernetes, cloud security, cloud infrastructure, and CI/CD pipelines. Our managed and consulting services are a more cost-effective option than hiring in-house, and we scale as your team and company grow. Check out some of the use cases, learn how we work with clients, and read more about our DevOps-as-a-Service offering.