Azure Landing Zones Implementation

Chapter 2: Our heroes discover the great wizard Terraform

The story so far

Our team of heroes discovered the magical land of Azure Enterprise Scale Architecture and heard about Landing Zones. With the insurmountable task of implementing what they learned, they set out to talk to the great wizard Terraform, a mythical being said to be imbued with enormous power.

Need a longer recap? Check out the previous article in the series: https://www.dhirubhai.net/pulse/kickstart-your-azure-landing-zone-deployments-terraform-shiftavenue-onkwe

Tools of the trade

To wrangle configuration code for policy-driven infrastructure, a host of good tools is essential. The next paragraphs contain the opinionated tools that I use often, with joy and great success.

Source code management

Arguably the most important tool for lack of a better word is a proper source code management solution. Git is the de-facto standard for this task, so I strongly suggest you use it. What hosting provider is the best? That is a harder question, as it will depend on several factors. For me, personally, the combination of GitHub and Azure DevOps has worked very well over the last years. Are those the only good tools? Absolutely not.

As a general rule of thumb: If you want strong collaboration with the community, GitHub is the best. If you don’t want people outside your organisation raising issues and commenting, i.e. if your repositories are private anyways, Azure DevOps is a cost-effective and versatile solution that includes superior project management in my opinion.

Enough about this boring stuff, and on to the good ones.

Editor and plugins

Having used many editors and integrated development environments over the years I am a happy Visual Studio Code user now. It supports most projects I am working on, which for me consist of PowerShell module development, light C# development, a little Ansible and Terraform code. Only if I work on actual C# projects that consist of more than a small helper class do I need the full Visual Studio.

For our needs though Visual Studio Code is a great starting point. Together with the extensions Terraform, Azure Terraform, GitHub Actions, and YAML you have a powerhouse for editing Terraform projects.

You’ve spotted YAML already - I said this was opinionated. When designing configuration data, I enjoy the flexibility and readability of YAML. Especially when compared to shudder JSON. You will later experience why I selected YAML. A hint: Auto-completion ??

Enacting the configuration code

The last tool in the toolbelt is the one that will enact your configuration. For the scope of this article, this means Terraform. In other projects, this would be PowerShell, Ansible, az cli or yet another tool.

The path to happiness: Management groups and policies

With the tools ready for action, our heroes set off to unknown lands. Battling myriads of App Services, Storage Accounts, AKS clusters, and more, they quickly stumble upon a magic nostrum: Management Groups and Azure Policy.

Having read the first article and the accompanying links to the Microsoft documentation, you already know how useful management groups are. They allow you to sort your subscriptions in neat little boxes like Connectivity and Identity subscriptions, Application Landing Zones, and more.

What’s even better is that combined with Azure Policy, management groups allow you to neatly and efficiently manage large swathes of your cloud footprint with little effort. I say little, but I can hear those of you who had the pleasure of creating JSON policies already wincing. But we’ll get to that.

Before we can get started with the code you’ve all been waiting for, let’s take a moment to review our target structure.

We are concentrating on our management group structure and its associated policies first of all. The rest will fall into place, I promise.

If you search the Terraform registry, the Microsoft documentation, or the internet in general, you will find lots of ways of deploying a management group structure. What I always focus on, in particular when it comes to resources that are somewhat global or organisation-spanning: Scaling and Infrastructure-as-code, or policy-driven infrastructure. What I do not want is a bunch of scripts that are never to be touched and that only work in laboratory conditions.

That being said, the approach should still be flexible and not consist of a single, large, cluttered file - be it an ARM template, a Terraform module, or a DSC configuration. Modularity gives you flexibility and shifts the focus on good configuration data.

Sauron’s eye is on the move: Terraform module “Azure Enterprise Scale”

To start with a management group structure, I - apparently the Sauron in this setup - would like to highlight one of two modules we will mainly be using. It is the lovely https://github.com/Azure/terraform-azurerm-caf-enterprise-scale module, which deploys the essential bits and pieces for your enterprise-scale cloud footprint.

As I mentioned earlier, we will concentrate on the core resources first. In the next article, I will highlight the connectivity portion a bit more, as well as introduce you to the landing zone vending machine - a brilliant little module that we can make even better using cleverly-crafted configuration data.

Deploying even the very basic samples available in the repository, you will already get a very well-configured landing zone infrastructure. As long as you do not need to change any settings, the resulting code will look neat enough.

As soon as you start going through the code and discover custom settings that you might need, such as the networking configuration, the code will get less and less tidy and manageable. At this point, I promise you a huge quality-of-life improvement if you adopt a simple way of working: Separating configuration data from the actual configuration!

If you’ve used Terraform for a while already, you are already familiar with the concept. After all, this is what variables are for. Yet for some reason, people love to hard-code things instead of crafting data structures that they can feed their configuration code. Don’t be like them!

So, after being given the configuration data treatment, a configuration could look as simple as this:

Can you spot any hardcoded data apart from the library path and the custom landing zone configuration? No? Good. Prepared in the previous code sample you can also find the strict subscription association turned off. This has to do with the Landing Zone Vending module we will see in the next article. We want the Landing Zone Vending module to control the management group association of our subscriptions.

Configuration Data meets Configuration

To craft the necessary configuration data for any project, think first of what you are going to need, and how many instances of it there might be. For my customer project that sparked this article series, the premise at first was one Entra ID tenant with separate Identity, Platform, Connectivity, and Management subscriptions plus a potentially limitless number of application landing zone subscriptions.

The structure in my head already took shape when hearing these requirements. Lists of objects, for each object a piece of code needs parameterisation. While the CAF module already lends itself to streamlined operations by having users specify a root ID, we have a lot of copied source code if the same project is used for multiple tenants. Granted this is a special problem to have, but I always try to think ahead.

A good format for configuration data as a result of my past projects is YAML. Like JSON it is a terse format for serialisation and deserialisation of data. While JSON always hurts my eyes with its many quotes, YAML looks a bit nicer. It might still not be the best, but try it first.

Since our core configuration consists of several components, we simply throw in one configuration file each. For my purposes, it was always an adequate approach, but with a module this complex, the configuration data looks daunting at first glance.

This is where schemas can come in handy. Some of you who have been around long enough or regularly develop things know them already. If you’ve ever written an Azure Resource Manager template, you will have seen a schema reference. The schema describes how your data is supposed to look like. For both JSON and YAML we can use JSON schema (even if written in YAML ??) as seen on https://json-schema.org/.

Why go through all the trouble though? The answer is quite simple: IntelliSense, auto-completion - those quality-of-life tools that help you when editing or creating those files. Oh, and validation of course. But that doesn’t make writing content easier.

While GitHub Copilot also inserted itself in the screenshot, you can see in the fly-out that our property was picked up as intended. Now, if you ignore the data types and enter wrong values, a good editor like Visual Studio Code will of course highlight this immediately. Neat!

If you browse to our template repository at https://github.com/shiftavenue/caf-configuration-data, you can find all three main configuration files ready for you to fill out. This is already more than enough to get started with. So go ahead, click the big button that says “Use this template”!

Stately state management

So far, so good. Using Terraform, you can deploy the entire project easily. To recap how this would be done, in case you haven’t used Terraform yet:

• terraform init
• terraform plan -out theplan
• terraform apply theplan

Now, as you are working in a repository, chances are you are not working alone. Think about the chaos that would ensue if everyone did the init, plan, apply dance themselves, possibly at the same time?

This is where the Terraform state comes into play. If you ran the previous commands, a local state file will have been created. This state file contains the entire resource tree and the current state of all resources. Terraform then compares current and actual state on subsequent planning runs.

While operations on the state are running, whether it is a “plan” or an “apply”, the state is locked. This means that there can be no concurrent access. This reduces the potential for misconfigurations.

Lucky for us, we can collaborate using the state. While Hashicorp offers cloud state storage, you can also use other so-called backends. Since we are deploying Azure landing zones here, we have chosen an Azure storage container. The storage account containing it is configured with blob versioning in order to revert back states. This can be very useful while troubleshooting.

In the template repository, the Azure Resource Manager (“azurerm” in Terraform) backend is already included in the code but needs configuration. When we deploy using a pipeline in the next chapter, we will revisit the state, and in particular, the permissions required for the remote state to work.

For now, keep in mind that the state file is crucial for Terraform to work properly.

Next up

In the next chapter, our heroes take a trip to the Bazaar of Landing Zone Vending and are victorious together through the magic of DevOps.