Apigee Continuous Integration & Deployment (CI/CD)

CI/CD reduces integration issues during deployment and increases the speed of finding and fixing code errors. The automated build verification step ensures the system is working as expected after many frequent integrations and helps building confidence in the code at every stage of development. API Continuous Integration can be enabled by the following steps:

1. Source control
2. Branch push and pull/merge requests triggering automated builds
3. Static code quality analysis
4. Unit testing
5. API proxy deployment to an Apigee environment
6. API documentation updates

Source Control Management (SCM)

Source control is an integral part of any software development project. It is where all source files, configuration, commits and history live and therefore it is where most of the integration happens between multiple developers in a team.

All commits for a particular project must be made against a single main branch within a repository called master branch. All the pull requests comes to master branch from its branched nodes called feature/bug_fixes branches. Based on the project requirements & complexities as many required branches are created and maintained.

Source control is the single source of truth in terms of all code, configuration and also being used to store packaged project artifacts. Once the artifact package constructed from SCM, it can then be reused in subsequent build iterations as a ready-made package bypassing initial package creation from SCM.

Developer(s) within a team start pushing their feature local branch (host system like laptop/desktop) containing a number of commits to the repository (origin).

Static Code Quality Analysis

Apigee has the functionality to execute custom code written in Java, JS and Python placed in any flow (request or response). It also has the functionality to execute node.js code placed as a target to the API proxy. It is recommended to run static code analysis for these custom code to promote consistency and language specific best practices within the team.

There are various tools available to do this for all of those languages mentioned. Most of them allow editor integration and source control hooks that can be adopted in the development environment. Most of the popular task runners offer watchers that can be used to trigger such analysis on changed files only. Run quality analysis on code for local development.

When it comes to integration though we shouldn’t rely solely on local analysis but need to integrate such tools into our continuous integration process so that the process is automated to catch any quality issues if they are missed in the local development environment. Our recommendation is to choose a CI tool that can run the code quality analysis and fail the build fast if it finds any problems.

Code analysis tools come with predefined rules that checks various aspects of the code. Team may not agree with some of the rules or conventions that tool is enforcing but nowadays most tools have the option of disabling some of the rules which can be configured as team’s conventions.

Unit Testing

There are two main areas where it is recommended to write unit tests for Apigee API proxies:

1. Scenarios where it is not possible to test the entire API proxy only with integration testing. These are operations that are hidden from the API client, e.g. JS callout to another API.
2. Testing isolated, complex but crucial code, e.g. JS code that does complex IP blacklisting using various rules, encryption, signature generation

Test coverage is especially important for the second area mentioned above. Integration testing on its own also will not give any coverage indicators. If relied on code coverage for such complex code, unit testing is the best bet.

API Proxy Deployment

The basic means of deploying an API proxy is via Apigee Management API. All other tools, including Apigee Management UI are using APIs to deploy a proxy. Apigee proxies can also be deployed by command lines.

There are also various tools that are built within the community as open source projects to help with deployment:

• Apigee Maven Plugin
• Apigee Maven Proxy Dependency Plugin
• Apigee Grunt Plugin
• Apigeetool

The recommendation is to integrate one of the above deployment tools with the CI to perform API proxy deployment.

API Documentation

As API proxy interfaces are evolving, their documentation must also be kept in sync. Apigee Developer Services Portal has a feature called SmartDocs which lets to document all APIs in a way that makes the API documentation fully interactive Or Confluence can also be used to achieve it with standard templates which are not so interactive as SmartDocs.

Interactive documentation means portal users can:

• Read about APIs
• Send a live request to API
• View a live response returned from the API

Represent the APIs by creating a model using WADL or OpenAPI (formerly Swagger) specification which can be modified during development and pushed to Apigee Developer Services Portal from CI using APIs so that the documentation is kept up to date with the API proxy interface.

API Configuration

A typical Apigee deployment includes modifications to environment configuration together with policies and custom code. These include changes to KVM, cache resources, target servers, products, applications, keystores, truststores, and others. Apigee exposes management API resources that can be used to manage environment configurations.

The recommendation is to automate modifications to Apigee environment configuration during CI builds using selected Management API resources or deployment tool that is required.

The CI benefits include even though there are other tools to automate API deployment:

• Configuration stored and maintained in SCM (single source of truth). Configuration changes subject to usual change control procedures (pull/merge requests, approvals). Changes implemented in a feature scope also include configuration modifications.
• Changes applied in an automated fashion eliminating manual steps to trigger/stop deployments, mistakes and forgotten entries. 
• Audit of changes kept and managed through source control. Existing Apigee audit reporting do not include all configuration areas.
• Apigee does not keep backups of your environment configuration. If configuration is lost (mistakes, org/env removal, etc), then need to reconfigure the environment from scratch without a master copy. Keeping configuration in source control and applying them automatically will save time and keep things well-balanced in these scenarios.

Continuous Integration Process

The overall process for continuous integration for Apigee proxies looks like the following:

• A local feature branch is developed, tested, committed to SCM. It is pushed to origin.
• CI automatically picks up the change in the origin feature branch and starts the activities explained in benefits above to validate and deploy the implementation in development environment.
• Once changes are validated, the developer creates a pull/merge request to merge them into master (Or based on branching strategy). Changes are accepted and merged after review process.
• CI automatically picks up the change in the master branch and starts the activities explained above (like feature testing, coverage limit expected, code analysis, performance, acceptance and other testings) to validate and deploy the integration in development environment.
• Once integration is validated, CI automatically deploys the new master branch into integration or staging environment.
• From here after, changes move through the environments up to the production environment. Each environment progression starts with that change merged with the target environment branch manually or automatically based on test strategy. The recommendation is to rely on automated testing/verification instead of pausing the progression for manual testing or interventions in order to make the whole process as agile and fast as possible. These continuous/frequent integrations will move towards continuous delivery to production.

This article mentions some of the tools that are used by Apigee Customer Success team however it doesn’t necessarily recommend those tools over others. The emphasis should be on the capabilities of the tools & the adopting CI/CD rather than the tools themselves.

Conclusion:

Overall, effective application of Continuous Integration practices can provide greater confidence in producing software. With every build, the team knows that tests are run against the software to verify behavior, project coding standards are being met, and the result is a functionally testable product. Since Continuous Integration can inform you immediately when something goes wrong, developers and other team members have more confidence in making changes. As Continuous Integration encourages a single-source point from which all software assets are built, there is greater confidence in its accuracy.

References:

• Source code management branching strategy for API
• Unit testing code coverage
• Unit testing API Proxies developed in Apigee Edge using mocha.js
• Deploy API proxies using the management API
• Deploying proxies from the command line
• Using SmartDocs to document APIs
• Apigee Tools & Plugins - Apigee Development Made Easy