Advanced Azure Pipelines Using YAML

In previous posts, we built a pipeline by creating jobs, tasks, and triggers. In this post, you'll learn how to customize an Azure pipeline using YAML. This includes creating condition statements with variable groups to set complex conditions, offering more flexibility compared to the classic online version. For example, YAML allows you to deploy mobile applications simultaneously to both the Google Play Console and the App Store Connect.

By the end of this post, you will have learned how to create both build and release pipelines using YAML. Additionally, we will also know how to clone, export, and import YAML configurations from the classic editor in the Azure DevOps portal.

This post will cover the following topics:        

- Creating a build pipeline using YAML

- Creating a release pipeline using YAML

- Cloning, exporting, and importing a YAML pipeline

- Complex YAML configurations

Let's begin by creating a pipeline using YAML syntax.

Creating a Build Pipeline Using YAML

In this section, We'll learn how to build a pipeline using YAML, view the YAML configuration on the Azure DevOps portal, and save the YAML file in Azure Repos. To create a build pipeline using YAML, follow these steps:

1. Log in to the Azure DevOps portal, select your organization, navigate to the Pipelines page, and then click on "New pipeline."

2. Click on Azure Repos Git, which is a source code repository for the demo:

3. Click on the PacktAzureDevOps repository:

4. If you already have a YAML file, select "Existing Azure Pipelines YAML file." However, since we are creating a new one, click on "Starter pipeline."

5. Click on Save, and you can review a new pipeline in YAML format:

6. Enter a commit message to help you remember the changes made in the file, and select the "Commit directly to the main branch" option. This will save your file to the main branch.

If we would like to save your file in a new branch, then select Create a new branch for this commit. Click on Save.

7. After you click Save, you will be taken back to the main dashboard result of the build pipeline:

8. Click on Run pipeline:

After that, you can see a summary of the build pipeline results:

9. You can edit the pipeline by clicking on the ellipses (…) next to the Run new button shown in the following screenshot and then Edit pipeline:

10. You can now view the basic structure of the YAML file. Let's break down each part of the example YAML file:

A. The main branch of Azure Repos stores the YAML file.

B. This is the repository name.

C. This is the YAML filename.

D. The build pipeline will trigger any changes to the main branch.

E. The build pipeline will run on the Ubuntu operating system.

F. A script task containing a single line.

G. A script task containing multiple lines.

These components are illustrated in the following screenshot:

11. We can see the result after running the pipeline based on the YAML file in the previous screenshot by clicking on a job:

12. Click on Run a one-line script, which will display the Hello, world! Text. This is an example of when you would like to display the message inside the task of the Azure pipeline:

Creating a Release Pipeline Using YAML

In this section, we'll learn how to create a release pipeline using YAML, including how to define stages, jobs, and tasks. Follow these steps:

1. Edit the existing pipeline by clicking on the ellipsis (…) next to it, then select "Edit pipeline."

2. Replace all contents of the existing azure-pipelines.yml file, as shown in the following screenshot:

There are two stages, as shown in the preceding screenshot:

• The first stage will display the Build stage job
• The second stage will display the Release stage

3. we can validate the syntax of the YAML file by clicking on … next to Save and clicking Validate:

4. We will see the following message if the YAML file is valid:'

If the YAML file is invalid, We will see the following error message explaining which line has a problem:

5. Click Save | Run pipeline. We can see the result of a pipeline contains two stages:

6. We can rerun a specific stage by expanding it and clicking on the Rerun stage:

Cloning, Exporting, and Importing a YAML Pipeline

In this section, We'll learn how to clone, export, and import a YAML pipeline from the Azure DevOps portal. These actions help you save time by duplicating and adjusting existing templates. If you need to create a new Azure pipeline, you can do so by cloning an existing one. Here are the steps to perform these tasks:

- Clone: You can quickly clone a pipeline by copying and pasting the YAML file.

Export and import: The following steps show you can export an entire YAML file from a pipeline:

A. We can export a YAML pipeline by clicking on Edit:

B. Click on … | Download full YAML to download a file:

C. Open a downloaded file and copy and paste it into the new pipeline you created.

Complex YAML Configurations

YAML syntax supports various complex configurations that allow for the modularization and reuse of YAML files. Examples include template reuse and the implementation of template expressions. We'll explore these features in the following sections.

YAML Template Reuse

For large projects with multiple applications developed by the same team, defining common templates to reuse is beneficial instead of writing everything from scratch for each application's CI/CD needs.

Azure Pipelines supports referencing templates to reuse steps, jobs, and stages, which reduces YAML duplication since all applications or deployment processes within the project are the same. Additionally, you can include parameters in the templates to pass values that customize the behavior of the referenced template.

Let's consider a scenario where an Azure pipeline builds the same application twice with two different build configurations in the .NET language. The file in the following screenshot defines one parameter, buildConfiguration, and three steps: installing the NuGet tool (NuGetToolInstaller@1), restoring NuGet dependencies (NuGetCommand@2), and building the solution with the Visual Studio build utility (VSBuild@1).

You can see in the following example how the same dotnet-build-steps.yml file can now be used to build the same application in two different agents, Linux-latest and windows-latest, with the same steps and having the possibility to add other tasks before and after:

This feature offers great flexibility by reducing duplicate code in your YAML pipelines and promoting standardization across pipelines, which helps minimize errors.

More complex configurations can involve storing all templates in a separate repository, managed by another team responsible for assembling these building blocks. This approach supports the teams handling CI/CD pipelines, as illustrated in the following screenshot:

YAML Template Expressions

Expressions in Azure Pipelines provide a custom syntax that allows you to dynamically resolve values during runtime, acting as control logic in your templates. While there are many types of expressions, it is important to know that you can:

- Evaluate literals and variables.

- Use built-in functions such as coalesce, contains, eq, format, and many others to evaluate logical conditions or transform values.

- Use built-in functions to evaluate the job status.

- Use conditions to conditionally insert variable values or tasks.

- Loop through parameters with each keyword.

- Evaluate dependencies on previous jobs or stages, such as status or output variables.

The following screenshot shows an example of how expressions can be used to build and test an application using two different sets of tools (the msbuild or dotnet CLI tools), selecting the tool based on a parameter:

This post taught you how to create build and release pipelines using YAML. This method is more powerful for developers than using the classic editor on the Azure DevOps portal, as it allows them to save YAML files in their Azure Repos, facilitating review of each revision’s pipeline. YAML-based pipelines enable more efficient, transparent, and developer-centric CI/CD processes.         

You also learned about handling complex scenarios, reducing YAML duplication, reusing templates, and adding dynamic behavior through expressions. Additionally, you explored the advantages and limitations of YAML-based pipelines.

In the next chapter, you will delve deeper into implementing a build and release pipeline using YAML, and learn how to reuse build tasks using Node.js, NPM, .NET, and Docker to build a comprehensive pipeline.

Microsoft Azure DevOps Microsoft Azure Microsoft Learn