Azure DevOps for IBM ACE

I have put together a simple guide for ACE users to automate your application deployments using Azure DevOps. The main motive behind doing this is that there are not a lot of articles about CICD pipelines for IBM ACE deployments as it is not a widely used open source tool.

When it comes to automating any deployments there are myriad of ways by which you can lay down the solution. In this article I intend to use Microsoft Azure DevOps (ADO) as the platform of choice. This is greatly beneficial for projects that are being managed using Azure boards.

As opposed to a traditional automation server, ADO provides you with two pipelines that allows us to effectively organise two main automation activities- Build and Release.

Solution

This is a distributed architecture solution using Azure Repos as the code repository. Updates made to the Azure Repos will trigger the build pipeline to create BAR files using agents having ACE runtime. The newly generated bar files will trigger the Release pipeline configured with Target servers as the agents to deploy the application bar files to different servers or environments.

Build Pipeline

A Build pipeline is where we create the bar files for ACE application. This is done by creating a custom yaml file that has the build configurations as separate tasks. Here we can also configure the build server that is dedicated for build activity by making use of agent pools. Agents in ADO are servers that can be registered with ADO to manage workloads. An agent can be configured by running agent scripts within an agent server by using a PAT (Personal Access Token) token that can be scoped within ADO for access granularity. Once an agent is configured, this can then be linked to a build yaml pipeline to create the bar files and publish it to the build job. It can also be parametrised so that we can create a template to re-use the YAML file for any new build pipelines. Below is a sample ADO build pipeline for ACE running in Mac OS.

Main Tasks mentioned in above build yaml:

• Trigger: Here we mention the code repository location in Azure Repo to invoke this build pipeline. Any commits made to this repo will trigger this pipeline to generate artifacts which in our case are bar files. When it comes to code commits, its recommended to use a trunk based strategy ie, by creating an ephemeral feature branch, a pull request, and an appropriate merge to main branch upon reviewing the changes.
• Pool: Agent pools in ADO is where we can register as many agents as we require to manage workloads. As for this build activity, here we can mention the agent that is responsible for running mqsi commands and creating the bar files. An agent can be either a Self-hosted or Microsoft-hosted. It is upto us to decide on what agent type makes more sense to our projects. Allocating a Microsoft-hosted agent would mean whitelisting too wide an ip range for ADO to interact with agents that are spun up in different locations around Azure Data centres. For an ACE build we require an ACE runtime which means we are better off using a self-hosted agent as this would reduce the downtime for spinning up a build server that meets our demand. Also a point to note is that if need agents to run multiple pipelines then we need to purchase an ADO tier that allows multiple parallel-job runs. In a private ADO project, by default only one parallel job can be run.
• Stages: Different jobs and tasks can be logically grouped in different stages. Here we have a single stage for Build.
• Task: There are many out-of-the-box tasks available in ADO to enhance the pipeline and enrich the build functionalities. 'PublishBuildArtifacts' is a task that publishes the newly generated bar files and make it available for the release pipeline. We can always go back to these published artefacts by going through build history. In ADO we can configure Retention policies to retain the build so that we have the artifacts stored to a maximum of 365 last runs. If we require more than 365 then we can configure a retention lease.

Release Pipeline

Release pipeline in ADO is GUI based and lets you create a pipeline using box diagrams as different stages. Each stage can be configured with the target server where bar files are to be deployed using aforementioned agents. Trigger to this pipeline can be made continuous by enabling the appropriate option and then it will use latest published artifacts as a trigger to invoke release pipeline. Below is the release pipeline for ACE deployment.

Release pipelines also allows us to configure Pre/Post-Deployment conditions using Gates and Approvals. This way we can review the changes or look for a condition before a code moves into production. This also allows us to streamline the application lifecycle such that it meets organisational policies and standards. Retention Policies and Lease options also applies to the release pipelines so that we can retain the Release runs. Based on the requirement we can seamlessly make different versions of the code available to test teams by invoking corresponding release from release history. ADO also allows us to associate a change with a work item so that it can be traced all the way to releases. This end to end traceability is a power feature in ADO that gives us clarity on each individual releases and the associated code base for it.

Enhancements and other options

• Security - Sonar cloud can be easily integrated with this solution to perform a static code analysis before the build is generated. It gives all major code issues and security vulnerabilities that are there in the code base. Based on the quality assessed by Sonarqube we can stop the build early on and it gives us an opportunity to fix issues at an early stage and release the application to keep security vulnerabilities at bay. There are many open source tools out in the market that can be used to scan the code base to check for license issues, OWASP standards, and open source libraries used etc.
• Monitoring- As part of the release pipeline we can also include the mqsiflowmonitoring command such that application events can be emitted to the default MQ topic. These events can also be integrated with any popular tool like Prometheus using which these events can be pulled out of the MQTT server and then be visualised using Grafana dashboards.
• IaaC - Azure ARM templates can be used to provision Azure resources for IaaC model and can also be used to write pipelines in it. IaaC does not much much sense for companies that use on-prem servers for build and releasing application to target servers. In a microservice world where containers and K8s are required, it makes more sense to use Terraform or ARM templates to provision resources for development activities. This way we can use Source to Image pattern as part of DevOps solution, ie. Using official IBM ACE image as a base layer and adding another layer to it by copying the generated bar files to ACE server location and then pushing the new image to a private container registry. Every new changes will provide a new tagged image in the container registry from where a new container can be spun up or can be used in manifest files to create pod replicas in Kubernetes.

Conclusion

Using Azure DevOps for IBM ACE deployments is a no-brainer for Projects that are managed through Azure boards and companies which are invested heavily in Azure space. ADO comes with many rich features that allows us to come up with a fully fledged DevSecOps model that can abstract a lot of activities from developers and provides a very agile form of working in projects. As for this article I have merely skimmed the surface of what Azure DevOps is capable of when it comes to laying out a solution. There are loads of other features and capabilities I haven't discussed here for the simplicity of this article. I hope I was able to help tech enthusiasts in getting an idea on IBM ACE automation using ADO as the platform of choice. I highly recommend preparing for AZ-400 Microsoft DevOps expert level certification to learn more about Azure DevOps and DevOps in general.