Day 10 of #100DaysOfLearning

Day 9 focused on Lean and Agile with a greater focus on the People and Process aspects of DevOps.

And let's not forget that there is another perspective, the Technology perspective.

Today I learn about Continuous Integration and Continuous Delivery with a focus on the Technology and Process perspective.

CI/CD in DevOps

Continuous Integration (CI):

• CI involves automatically building and unit testing an entire application frequently, ideally on every source code check-in.
• Frequent check-ins aim to avoid long-running parallel code branches, reducing messy merge efforts and validating ongoing functionality.

Key CI Practices:

• Developers ideally build and test applications on their local desktops before submitting to a shared build system.
• CI focuses on shortening the feedback loop for every change, emphasizing frequent validation of code changes.

Continuous Delivery (CD):

• CD extends CI by involving the deployment of every change to a production-like environment.
• Automated integration and acceptance testing occur after build and unit tests.
• Small-scale deployment on a single server with mock environments or containers helps mimic a production environment.
• Testing concludes with a deployment to a production-like environment to validate the deployment process, functionality, and performance.

Continuous Deployment:

• Continuous deployment takes CD further by automatically deploying every change to the production environment.
• Despite potential fear, continuous deployment, when supported by trustworthy tests, removes manual gates for a continuous flow to production.

Implementation Considerations:

• Continuous deployment may feel intimidating, but it's not mandatory for all products, and DevOps implementation doesn't require it.
• Teams can benefit significantly from CI and CD, even if not pursuing continuous deployment.
• While the first DevOps team might not have adopted continuous delivery fully, subsequent teams have realized substantial benefits.

Build and Release Challenges:

• Some build and release processes involve giant releases with hundreds of changes piled up for months.
• These processes often require days spent on merge conflicts and manual workarounds, resulting in inefficient practices.
• CI/CD represents a significant improvement for products and teams, with continuous delivery offering substantial benefits.

Continuous Delivery in DevOps

Let's dig deeper into the benefits of Continuous Delivery.

1. Empowering Teams:

• Continuous Delivery (CD) is viewed as an empowering tool for teams, fostering a cultural shift within organizations.
• The CD pipeline serves as a self-service system, making the software delivery process transparent and understandable to the entire team.
• Operational and trusted CD pipelines contribute to a significant cultural change, fostering better collaboration and reducing strains between development, operations, QA, and security teams.

2. Dramatic Decrease in Total Cycle Time:

• Continuous Delivery significantly reduces the total cycle time from code commit to production deployment.
• In traditional methods, this time was measured in weeks or months, but with CD, it's measured in hours or even minutes.
• High-performing organizations employing CD can achieve remarkable speed in delivering changes, providing quick market value.

3. Improved Security:

• Despite common perception, security actually improves with faster development cycles.
• High-performing teams integrating security into their work spend 50% less time remediating security issues compared to low performers.
• Continuous Delivery aids in making compliance audits smoother and more predictable.

4. Continuous Practice and Stress Reduction:

• In CD, there's a constant practice of deploying release candidates with each code commit.
• Unlike traditional methods with big, infrequent deployment meetings, CD eliminates stress associated with release dates.
• Regular, continuous releases turn significant deployment events into routine occurrences, reducing anxiety and unpredictability.

5. Increased Value Delivery:

• Continuous Delivery allows organizations to deliver more value by spending less time on unplanned work or rework.
• Teams practicing CD experience a 29% growth in feature development, redirecting time previously spent on remediation to creating new features.

Less time spent on code rework due to integration issues means more developer time available to add new features, contributing to added value.

Continuous Delivery, despite requiring time and investment, yields long-term benefits. The five near-universal benefits of CD include empowering teams, dramatic reduction in total cycle time, improved security, continuous practice, and stress reduction, as well as increased value delivery. While other benefits may vary by organization, these observed benefits are considered compelling and widely applicable in the DevOps landscape.

Build Pipelines in Practice

A build pipeline is a sequence of operations and tools that facilitate the process of transforming source code into a deployed system. The composition and flow of the build pipeline are crucial, transcending the specific tools used. Here's a breakdown of the components and stages within a typical build pipeline:

1. Source Code Repository:

• Tools like Git, Perforce, or Subversion store all aspects of the source code, including AppCode, deploy scripts, and infrastructure definitions.
• Ensures versioning, code safety, and facilitates collaboration among multiple contributors.

2. Build System:

• Monitors the source code repository for changes and triggers necessary builds.
• Popular build systems include Jenkins, Bamboo, TeamCity, and online services like GitHub Action, CircleCI and Travis CI.
• Builds code, runs unit tests, and provides feedback and visibility into the build process.

3. Compilation and Build Orchestration:

• Language-dependent compilation tools or build orchestration tools (e.g., Make, Maven, Ant, Gradle) may be employed.
• Enables developers to build on local machines and keeps build configuration in source control.

4. Unit Test Harness:

• Verifies that a piece of code functions as intended.
• Basic tests with no external dependencies; may use mocking or stubbing for external systems.

5. Artifact Packaging:

• Packages the built code into artifacts based on the language (e.g., JAR files for Java).
• Higher-level packaging structures like RPM packages or Docker images may be preferred.

6. Artifact Repository:

• Artifacts are stored in repositories such as Nexus, Artifactory, Docker Registries, or storage devices like Amazon S3.

7. Deployment Tool:

• Deploys the service to test and production environments.
• Deployment code is treated as software, and issues are identified during the build and test cycle.

8. Testing Stages:

• Integration Testing: Exercises a real running service in a real environment to ensure correctness.
• End-to-End Acceptance Testing: Uses the same artifact and deployment tool for testing, ideally including production-like acceptance tests.

9. Additional Tools (Optional):

• Various tools can be added, such as test coverage measurement tools, linters, security testing tools, and performance testing tools.

Key Principles:

• Nailing the basics is crucial for achieving continuous integration and, if desired, continuous deployment.
• A well-designed build pipeline supports a culture of continuous delivery, fostering collaboration and minimizing issues during production deployment.

CI/CD best practices

1. Developer Responsibility:

• Developers are responsible for the successful build and deployment of their check-ins.
• Successful deployment is the responsibility of the developer, not just the build, QA, or ops engineers.

2. Small Changes and Build Quality In:

• Embrace the lean mantra of making small changes and building quality into the process.
• Making fewer changes at once and minimizing the delay in testing helps maintain code quality and makes bug detection and fixing easier.

3. Addressing Broken Builds:

• Avoid leaving broken builds behind, as it blocks the team and encourages the piling of untested changes.
• Engineers should not check in or merge to master when the build is broken, and they should actively contribute to fixing the problem.

4. Testing Discipline:

• High-quality automated testing is crucial for continuous delivery.
• Developers should run tests locally before check-in or have a branch that builds and runs tests before merging into master.
• Avoid flaky tests; log bugs and fix them promptly to maintain confidence in the testing pipeline.

5. Strategic Use of Manual QA:

• Reserve manual QA for high-value activities that are challenging to automate.
• Manual QA should focus on providing expertise in constructing good tests, building maintainable UI tests, validating performance tests, and conducting resilience testing.

6. Automation in Deployment:

• Automate deployments at every phase, from development desktop to CI deploys to production deploys.
• Treat every piece of the environment as code, continuously build and validate it, and eliminate surprises by ensuring environments are nearly identical.

7. Efficiency in Build and Deployment:

• Keep the build and deployment processes fast, and investigate if they take more than a few minutes.
• Implement parallel component builds and test runs, tune timeouts, and optimize build system parameters.

8. Testing Flow:

• Follow the testing philosophy of 70% unit testing, 20% integration testing, and 10% end-to-end testing.
• Maintain an efficient flow where build, test, and deploy processes are streamlined to enhance cycle time.

9. Source Control and Automation:

• Everything, including infrastructure build, tests, app code, and monitoring configuration, should be in source control.
• Automation should cover every aspect, and routine tasks should avoid manual interventions or executions without tests.

10. Discipline in Continuous Delivery:

• Stick to the discipline of not doing things manually or without tests to achieve successful continuous delivery.
• Routine check-ins should result in code being in production within minutes.

These best practices contribute to the efficiency, reliability, and success of a continuous delivery pipeline, emphasizing the importance of automation, discipline, and a focus on quality throughout the development and deployment process.

Next Action

As discussed in this lecture, pipelines are by no means unique. There should be one that works for you.

With that in mind, I will review CI/CD in a practical way using GitHub Actions.