CI/CD/CT - Shift-Left and Shift-Right with Automated Continuous Testing

This year marks 30 years since the Continuous Integration (CI) practice was proposed by Grady Booch in 1991 in the scope of object-oriented software development. Continuous Integration aims at developing and integrating software artifacts in an incremental way by merging all code changes into a shared repository as often as possible.?

Later on Kent Beck , who developed Extreme Programming (XP) and Test-Driven Development (TDD), adopted the concept of Continuous Integration and advocated integrating code more than once per day, even as many as tens of times per day thus significantly reducing the cost of fixing bugs early.

Continuous Testing

Continuous Testing consists of two lean components: Shift-Left and Shift-Right Testing. Meaning testing early (during requirement gathering, planning, development, etc), testing often, testing everywhere, including in production. Continuous Testing is a way to boost, speed-up, and support CI/CD pipelines.?

Larry Smith once mentioned, “Bugs are cheap when caught young.” He saw the harm that could be caused by testing software late and in 2001 introduced Shift-Left Testing. He wanted to bring testing teams closer to developers, operations teams, and product planning teams. Therefore include the mindset of quality and testing at an early stage of Software Development Life Cycle, and compel everyone to be responsible for quality. The quality ownership means better requirements, better design, better code, and better tests. It allows you to prevent, detect and fix bugs early.?

Automation is the most important part of Shift-Left Testing. If test team members are part of regular Agile standup meetings, they come up with automation frameworks and languages that are best suited for a particular application to execute unit and smoke acceptance automation tests close to code thus improving code health.?

Agile/DevOps Shift-Left Testing in Continuous Integration focuses on automated unit tests via TDD process to make sure the application is not broken whenever new commits are pushed into the main branch. All code must pass all unit tests before it can be released.?

Shift-Right is a method of testing and monitoring in production. This approach helps developers uncover new, unexpected scenarios that may not have been detected within lower environments. It therein elevates to fix bugs before end users find them and boost software efficiency. In conjunction with manual exploratory testing, it can also have a big impact on the product as a whole to achieve customer's success.

Continuous Integration

A successful Continuous Integration infrastructure relies on a dedicated server that can be on-premise or hosted via cloud platform providers. When the pipeline is triggered, the server in charge of fetching the code at stake is building it, preferably in an isolated way.?

The isolation can be done by virtualization technique:

• Hypervisor - cloud Virtual Machines (on AWS, GCP, Azure, etc.)
• OS-level virtualization - Docker Engine with isolated containers that share OS and/or bins/libraries

Docker was introduced in 2011 by Solomon Hykes . Docker enables developers/SDETs to easily pack, ship, and run any application as a lightweight, portable, self-sufficient container, which can run virtually anywhere. A container is more ultra-light than a Virtual Machine in terms of disk, memory, and CPU resources.?

In a modern web application, you can use Virtual Machines to deploy into different environments (DEV, QA, STG, Prod), while containers are best suited for packaging/shipping portable and modular software. The two technologies can be used in conjunction with each other for extra benefits. Docker containers can be created inside of Virtual Machines to make an ultra-portable solution.?You can spin up transient environments as needed, do the work and tier it down.

Kubernetes , founded by Joe Beda , Brendan Burns , and Craig McLuckie in 2014, is the container orchestration tool that is ideal for scalability of CI/CD pipelines. It can deploy Docker application containers in the cloud, schedule batch jobs, handle workloads and perform rollouts. Kubernetes maneuvers those operations using efficient orchestrating algorithms that perform well at scale.

Continuous Delivery?

Continuous Delivery (CD) as a concept was first mentioned by David Farley and Jez Humble in 2010 in their book Continuous Delivery: Reliable Software Releases Through Build, Test and Deployment Automation . However, the Continuous Delivery definition continued to advance and now has a more developed meaning. It is an extension of Continuous Integration to make sure that you can release new changes to your customers quickly in a sustainable way. It means that on top of having automated tests, you have also automated your release process and can deploy your application at any point of time with the click of a button.

With Continuous Delivery you can decide to release every few minutes, hourly, daily, weekly or whatever suits your business requirements. However, if you truly want to get the benefits of Continuous Delivery, you should deploy to the production as early as possible to make sure that you release small batches that are easy to troubleshoot in case of issues.

Continuous Deployment

Continuous Deployment (CD) is a release process that relies on automated testing to validate changes to a codebase from one environment into another. It either can be propagated structurally through all environments DEV-QA-STG-Prod or you can spin up a transient ephemeral?environment for a particular feature branch that could be tested, merged to master and deployed to Production.

The ideal state of Continuous Deployment is when all automation tests are triggered automatically as a part of the build pipeline in DEV environment. If the build is complete, then it would trigger to run the sets of tests. If the tests pass, then the build would deploy into the QA environment. And then the additional sets of tests would run automatically. If those tests pass, then it would deploy into the STG environment where an additional set of tests would run. Then depending on the project some builds would be pushed into Prod environment automatically or for certain area of application the manual sanity checks would be made for safety before deploying to Prod.

The reason is that by having automatic deployments, you would be able to focus your energy on improving your tests rather than having to periodically stop everything to coordinate a release.

To achieve such a state, you would need the Continuous Integration tool that fits your organization's requirements. Keep in mind the following criteria when selecting a tool:

• Hosting options
• Integrations support
• Containers support
• Usability design
• Library of reusable code

Here are the most popular CI/CD tools with times of introduction to the world:

1. Jenkins (2011)
2. CircleCI (2011)
3. Travis CI (2011)
4. Gitlab CI (2012)
5. Azure Pipelines (2012)
6. AWS CodePipeline /CodeDeploy (2014)
7. GitHub Actions (2019)

Continuous Monitoring

For CI/CD/CT efficiency ensure to monitor your automation scripts executions, code deployments and even your entire production environment with Slack , New Relic or Datadog tools. New Relic and Datadog help you monitor and receive alerts, if there is low throughput, high CPU utilization, high response time, high load time, slow web transaction time, etc. By integrating with Slack, results of each deployment and automation script execution would appear in particular Slack channels. Developers can then start threads to debug failed builds. Therefore they can eliminate performance bottlenecks and respond fast to any issues.

CI/CD/CT and Tools

Here is the diagram of suggested tools and frameworks of automation testing and types of testing methodologies in Agile SDLC with Continuous Integration, Continuous Deployment, Continuous Testing, Continuous Delivery, and Continuous Monitoring.

Conclusion

Continuous Testing with Shift-Left and Shift-Right Testing can dramatically increase your productivity to deliver high quality software with faster release cycles. It cuts down the feedback cycle and helps avoid the tedious task of rewriting pieces of code. Repeatedly checking different aspects of the Software Development Life Cycle from the earliest stages of development until delivery, promises to Quality Engineers, Developers, and DevOps/SRE teams that they are minimizing risk every step of the way.?

With Automation and Continuous Monitoring your team can have more confidence in the application resilience, and reliance in fast deployment practices, delivering Continuous Quality to your customers.