A Terrible Terraform Pattern

Here's a scenario I've seen in multiple enterprises using Azure.

1. Company decides to go with Terraform for all their infrastructure deployment automation.
2. Company doesn't want users to download the Terraform modules for Azure directly from the Internet. Several possible reasons: security might want to restrict some things from being used, folks want to "shift security left", or there may be performance concerns with using only the publicly-downloadable modules.
3. Company decides to create repos for each of the different Azure resources, each with a "custom" Terraform module that they control and version themselves.
4. Company tells and/or restricts all engineers to use only the "approved modules". Some call them "approved", some call them "golden", etc.

From a high level, this makes sense. It allows the security team to vet each of the resource types that the company is going to deploy. It allows the enterprise architecture team to restrict certain options from the engineers to keep from having features out that haven't been approved. It provides a team to point to for all of the future updates to the Azure provider, meaning that every application team isn't doing the same work to perform the updates.

Let me say this:

THIS COMPANY HAS BUILT A STRUCTURE THAT WILL CRUSH THEIR CLOUD AMBITIONS.

Now, let's assume that the company in question is the company where you work. Here's the crux of things: the Terraform modules for your chosen cloud provider are not released at the same time as the services the provider is offering.

When Azure releases a new service, they do not necessarily (and almost never) release the Terraform module for that service. The AzureRM module is not part of Microsoft's ecosystem, as it's an open-source project only. Microsoft does work on it and contribute, but they do not have to. Similarly, it is not maintained by Hashicorp (the makers of Terraform) either...again, they do contribute, but it's not "same day" as new Azure features happen.

This means there's a necessary delay between the feature in the cloud being available, and the Terraform module being available to set it up. Your application team will already be waiting for the module to be updated.

When you decide to create "your own" module, you then place a team in between the open-source module and your application teams. Your application team will now have to wait for your "Terraform team", too.

Your Terraform team is going to be exactly like any other team in your company. They will have a work backlog. They will want to manage their workload, just like your team does. They will have intake forms for you to fill out. They will want to have meetings to see what your new request is about. They may even decide they won't make the change because you are the only one of their many customers and they have other priorities.

Your application team is now stuck doing paperwork, sitting in meetings, and debating with people whether or not the feature is important. You can forget about consuming the service immediately, unless you do something like deploy it with a different method while you wait on the Terraform team (it will happen...you cannot restrict Azure CLI, for example). Which takes you down the path to Terraform state file hell.

DO NOT FOLLOW THIS PATTERN IN YOUR WORKPLACE.

The reasons to follow the pattern seem reasonable, but you are building a house of cards that will inevitably break down your application teams. Please remember that the Terraform modules ARE NOT the actual resources that are getting deployed. None of the work from your "Terraform team" is providing value to your actual customers...the team is just repackaging stuff that they literally download themselves (from the same place your application team would, as a matter of fact) and is causing your developers unnecessary delays.

Let teams download the Terraform modules from the source, which will be the quickest way for them to get to consume new cloud features. You can manage the security and compliance of your resources using things like Azure Policy, and not stop your application teams from doing their jobs. (As a matter of fact, in every case I've seen, the security teams still use policy to provide such compliance, which makes the Terraform team completely redundant in that respect.)