CI/CD Pipelines and Caching of Dependencies

Introduction:

In this blog, we'll be taking you through a brief explanation of CI/CD Pipelines and how to implement caching of Maven Dependencies in the pipelines while deploying your Mule application to CloudHub.

What is CI/CD?

CI/CD practice is one of the best examples of a great quote by Kent Beck: “Make it work, make it right, make it fast.”

Continuous Integration or in short CI is a software development practice in which the developer of the code would save (commit) their code in a central repository on a regular basis, Continuous Delivery or CD in short, refers to automating the whole release process by making use of automated pipelines.

Prerequisites for Azure CI/CD pipeline:

• Anypoint Studio to develop your application
• Azure DevOps account with access to hosted parallelism
• An active Anypoint Platform account
• Maven and Git installed on your local system

Tutorial:

1. The first step after you create you Azure DevOps account is to raise a request for hosted parallelism, else whenever you execute your pipeline you will get the below error:

To raise a request go to https://aka.ms/azpipelines-parallelism-request and fill the form with the correct organization name. It will take around 2 business days to get this request approved.

2. Now create a project in Anypoint studio which you wish to deploy on your cloudhub. For this tutorial we’ll take a simple example which consists of an HTTP Listener and a Logger component.

3. Now configure your project’s pom.xml to enable the deployment of your project on CloudHub via Maven. This uses mule-maven-plugin which is already a part of your pom.xml generally, in case it is missing just add the dependency and use the following configuration:

<plugin
<groupId>org.mule.tools.maven</groupId>
<artifactId>mule-maven-plugin</artifactId>
<version>3.5.4</version>
<extensions>true</extensions>
<configuration>
<cloudHubDeployment>
<uri>https://anypoint.mulesoft.com</uri>
<muleVersion>${app.runtime}</muleVersion>
<username>${username}</username>
<password>${password}</password>
<applicationName>${cloudhub.application.name}</applicationName>
<environment>${environment}</environment>
<region>${region}</region>
<workers>${workers}</workers>
<workerType>${workerType}</workerType>
<properties>
<key>value</key>
</properties>
</cloudHubDeployment>
</configuration>
</plugin>        

These variables can be defined at runtime, or can be directly given in the pom.xml, For further reading on these properties or any additional flags, you can visit the official documentation on https://docs.mulesoft.com/mule-runtime/4.4/deploy-to-cloudhub .

Remember to keep your application name unique, otherwise you’ll get an error while pipeline execution.

4. Be sure to check your Maven settings.xml file has all the correct credentials and the required repositories are added there, otherwise your pipeline would fail. This settings.xml file is present at your Maven home in your local system.

5. Now let’s create a repository on Azure Repos.

• Go to Repos in your Project -> Click on initialize.

• Now clone your repository into your local system by getting the clone URL from your Azure Repo.

• Now using this URL execute the git clone command on your local system to clone your repository.
• It will ask for your devops credentials while cloning. Alternatively, you may use git credentials for HTTPS/SSH according to your requirement.
• Now commit and push your code into the Azure repository it would look like this after successful code push:

6. We’ll now proceed to create the Pipeline:

• Click on Create pipeline and then go to classic editor:

• As we are using Azure Repos, we’ll select the Azure Repository here. You may proceed with any other option if you have already set up your repository somewhere else:

• Now click on Empty Job. If you have exposure to write pipelines using YAML to create and configure your pipelines you may select the YAML option but if you are looking for ease of use, just go with an empty job.

• If you have any specific agent requirements i.e. you want your pipeline to run on ubuntu/windows you may select that in Agent Specification. The default is windows-2019 which we are using for our demo.
• Now click on + Button to start adding agent jobs:

• We need to add 3 jobs, Cache, Maven and Download Secure File. The order should be Cache then Maven and in the end the DownloadSecure file, otherwise the Cache won’t work.
• For Cache, provide the below properties:

• We are providing pom.xml as our key, so that the cache would check for any changes in pom.xml and would download the dependencies in case there is a cache miss otherwise it would use the dependencies present on the Path specified. For us we have Provided $(MAVEN_CACHE) as our variable.?

For the first time execution of your pipeline, the cache will always be a miss.

• Now to provide values to the variable go to the Variables section and define the value. For $(MAVEN_CACHE) the value should be $(Pipeline.Workspace)/.m2/repository. This is the location where the maven dependencies are stored on Azure. $(Pipeline.Workspace) is a predefined variable.

• Also, remember to provide all the variables which you want to provide here itself which will be used in your maven command.
• We have an option of creating variable groups in case you want to create some secure properties, such as your username and password. You can provide them in Library->Variable Groups

• After this group is created you have to Link it in the Pipelines-> Variables section.

• Now go to Secure files in the Library and add your Maven settings.xml from your local system here.

• Go to your pipeline-> Download Secure File in your agent job and provide the settings.xml from the dropdown under Secure file. In the Output Variable section, provide a variable name and copy the full variable from the Variable list which will be used to access the secure file via maven command.

• Now go back to the pipeline and go to your Maven Task and provide the maven details such as the location of the pom file, your goals and the command.

7. You can enable Trigger as well so that you can leverage the Continuous Integration ability of your Pipeline. Whenever you push your code the pipeline will automatically Execute. Go to Pipeline-> Triggers and check Enable continuous integration.

8. Now click on Save & queue and your pipeline will execute. In case there is any error that will be displayed in the pipeline itself.

9. Now you may check the Runtime Manager on cloudhub, and if the job was successful and all the parameters were correct you should see the application there!

Happy Learning!

Do reach out to us if you have any questions/suggestions.

Rahul- [email protected]

Kushagra- [email protected]

Authors

Rahul Kumar (Senior Software Engineer), Apisero

Kushagra Kakkar (Senior Software Engineer). Apisero