Azure DevOps YAML Pipelines

Multi-Stage CI/CD YAML Pipelines are a fairly new feature in Azure DevOps. They offer some great advantages versus the classic GUI Pipelines. One of the biggest advantages is that your entire CI/CD Pipeline is now represented in a single YAML file that can be checked in and tracked with your version control system of choice. As with a lot of things I write about, this is an incredibly deep and complex topic. There's no way to fit every little minute detail into one article. So, this will be a fairly high level overview.

The prior Pipelines experience was built around the GUI editors found on the Azure DevOps website. You can still access and use these classic experiences if you wish:

GUI Build Pipelines can be found under: Pipelines > Create Pipeline > Use the classic editor:

GUI Release Pipelines can still be found under: Releases > New Pipeline

So, what's the new method? Simply create a file named "azure-pipelines.yml" and define your Build and Release Pipelines as 100% code!

Of course, this brings some challenges with it. The biggest being that you'll have to learn the proper format & schema for the YAML file. Microsoft has a great reference online, which you can find here. But, understanding the major building blocks of the YAML Pipeline will get you a long way there. I'll go over those building blocks below.

Stages

A Pipeline is made up of one or more Stages. Stages are major divisions in a Pipeline. Stages are logical boundaries in your Pipeline, which you can use to pause your Pipeline and perform various checks. By default, Stages run sequentially. Each Stage starts only after the preceding stage is complete.

Jobs

Stages are made up of one or more related Jobs. Jobs are a unit of work assignable to an Azure DevOps "agent". The agent ultimately performs the work included in the Job. The agent can be a machine hosted by Microsoft or a machine that you host yourself.

A single agent can only run one Job at a time, so if you only have one agent then your Jobs will run sequentially, one after the other. If you have multiple agents, and you have the required amount of Parallel Jobs, then multiple Jobs can be run on multiple agents in parallel. To control this, you can utilize conditions and dependencies. For example, start Job 2 only if Job 1 has completed successfully.

Jobs come in two forms, a regular Job and a 'Deployment' Job. Both types of Jobs can live in the same Stage. It's recommend to put your deployments steps into a Deployment Job versus a regular Job. Deployment Jobs provide two major benefits:

1. Deployment history: You get the deployment history across pipelines, down to a specific resource and status of the deployments for auditing.
2. Apply deployment strategy: You define how your application is rolled out. This can be rolling, canary, or run once.

Steps

Jobs are made up of one or more Steps. Steps can be a task to run or a script to execute. Steps are run sequentially, one after another. Before a step can start, all the previous steps must be finished (or skipped).

Azure DevOps includes a big library of built-in tasks which you can use. You can see the full catalog here. There are tasks for building/compiling your code, testing your code, installing required tools, using packages and package repositories, deploying your app, and many more.

If you have custom things that you would like to do that aren't covered by one of the built-in tasks, then you can always just create your own custom script and run that.

Summary

I think that's plenty of information for an introductory article. As stated earlier, this is an incredibly deep topic and I already have follow-up articles in mind for the future.

#azure #azuredevops #devops #yaml #azurepipelines #pipelines

Sources:

• YAML Schema
• Deployment Jobs
• Pipeline run sequence
• Catalog of built-in tasks