Diagram as Code with Python

As a Cloud Native Development Architect everything for me usually starts with a diagram. Sometimes I create the diagram informally on a whiteboard along with discussions with my coworkers, sometimes it starts as a doodle on a piece of paper, a napkin or something else where I try to put together various pieces of puzzle to solve complex problems. Often the next stage is to formalize the diagram in a tool like Visio or Draw.io or sometime similar where I have to represent what was in my brain in a formal manner to get a buy in from higher ups.

And I suck at drawing. I often find myself asking for help from people who excel at it. It simply boggles my mind how some people can create complex diagrams in a few clicks where as I struggle for hours. However I do excel at coding. So imagine my happiness when I found diagrams.

Diagrams is a python module that allows you to express your Infrastructure Diagrams as Code. It allows you to rapidly prototype any system and supports most of the major cloud providers and SaaS systems, on-premise nodes, programming frameworks and languages.

So why Diagram as Code ? It has all the advantages of Anything as a Code plus more.

• It can be version controlled allowing you to track changes over time
• Multiple people can collaborate and contribute making editing easy
• Developer friendly. If you like coding, specially in Python, you will love it.
• Uniformity. All your diagrams will look and feel the same.
• Portability. Anyone who has Diagrams installed can download the code for your architecture and generate the diagram in whatever format the want.

Installation

Before you install diagrams make sure you have graphviz installed as it is used by diagrams to generate the images. Also make sure you are running Python 3.6 or higher.

After that use whatever tool you use to install python modules like pip, pip3, conda to install diagrams. e.g.

$ pip install diagrams

For anaconda use the command

$ conda install -c conda-forge diagrams

Examples

Let's see some simple examples.

from diagrams import Diagram
from diagrams.aws.network import APIGateway
from diagrams.aws.compute import Lambda
from diagrams.aws.database import Aurora

with Diagram("Lambda Invocation", show=False):
    api_gw = APIGateway("API Gateway")
    my_lambda = Lambda("Lambda")
    my_db = Aurora("Aurora Database")

    api_gw >> my_lambda >> my_db

Save this code as lambda.py and run it. It should generate an image in the same directory called lambda_invocation.png. The show=False means the image is not opened immediately.

Let's try a GCP example.

from diagrams import Cluster, Diagram
from diagrams.gcp.compute import GKE, GCF
from diagrams.gcp.network import LoadBalancing
from diagrams.gcp.database import SQL
from diagrams.gcp.analytics import PubSub

with Diagram("GKE Cluster", show=True):
    lb = LoadBalancing("External Load Balancer")

    with Cluster("Default"):
        pubsub = PubSub("PubSub")
        with Cluster("Internal"):
            gke = GKE("GKE")
            lb >> gke >> pubsub
        with Cluster("Backend"):
            mysql = SQL("MYSQL")
            gcf = GCF("Functions")
            pubsub >> gcf >> mysql

As you can see from the code we have a GKE Cluster inside an Internal Network fronted by an external load balancer which dumps messages into pubsub which invokes some Google Cloud Functions which in turn access MySql Database. Definitely won't win the architecture of the year award but a few lines of code generated the below diagram. Also show=True means the image was opened as soon as the script was executed allowing for instant feedback.

Finally let's see a pure Kubernetes example.

from diagrams import Diagram, Cluster
from diagrams.k8s.clusterconfig import HPA
from diagrams.k8s.compute import Deployment, Pod, ReplicaSet
from diagrams.k8s.network import Ingress, Service

with Diagram("Exposed Pod with 4 Replicas", show=True):
    net = Ingress("releasemanagement.org")
    with Cluster("Kube Cluster"):
        net >> Service("svc") >> [Pod("pod1"),
                                  Pod("pod2"),
                                  Pod("pod3"),
                                  Pod("Pod4")] << ReplicaSet("rs") << Deployment("dp") << HPA("hpa")

This example is a slight variation on the official K8s example from the Diagrams site because there is only so many ways you can show K8s pods and replica sets. Here we see 4 pods managed by a replica set and a HPA and fronted by service and Ingress controller. I added the Kube Cluster to show how easy it is to show clustering with diagrams although technically I should have shown the ingress controller inside the cluster. Here's the final result.

This is just the tip of the iceberg of what you can do with Diagrams. However as good as diagrams is at representing your architecture, it cannot generate or control your architecture. Which means there is no direct link between diagrams and Terraform, CloudFormation, Ansible, Chef, Puppet or any other tool that will allow you to orchestrate the architecture.

All the code in this article can be found on my github over here. For more examples with Diagrams go over here.