How To Build A Deployment Pipeline
In this article, I describe how to start, improve and speed up your Deployment Pipeline, with links to some FREE advice and training, and other useful resources.
The Deployment Pipeline is a very well-known concept in the IT industry today - although it’s not always well understood. It is not just about ‘deployment’ for a start! Probably my fault! - when Jez Humble and I were working at Thoughtworks back in the early 2000s, and doing the work that led to our best-selling book on Continuous Delivery, we developed the idea of a pipeline that organises all the software development activities needed to get from ‘idea’ to ‘valuable software in the hands of users’: a kind of machine with automated steps running in parallel, from commit to releasable outcome as quickly and efficiently as possible. This parallel processing approach means that we can deploy quality software repeatably and reliably. At the time, this reminded me of the ‘instruction pipelining’ in intel processors - and so we coined the term Deployment Pipeline, often referred to as a Continuous Delivery Pipeline.
This is my diagrammatic representation of a simple Deployment Pipeline.
The objective of a Deployment Pipeline is to be definitive for release. If the Pipeline says everything is good, then the change is ready to release without any further steps. To be complete therefore, a Deployment Pipeline must include any and all steps that are required to determine the releasability of for new software, i.e: all unit tests, acceptance tests, validation, integration, version control, sign-offs and any other tests or requirements to achieve releasability.?
That sounds like a massive undertaking! So where do you start??
Getting Started
Start by building a Walking Skeleton and building just enough Pipeline to support it - A “Walking Skeleton” is a tiny implementation of the system that performs a small end-to-end function.
Take a really simple use-case, or story, which involves the bare bones of your system and implement the Deployment Pipeline for this simple example:
We build this minimum example app, and use its construction to guide the creation of the simplest Deployment Pipeline that will serve our needs, and help us establish how we will work in future - TDD, Acceptance Tests, Auto Deployment - Continuous Delivery!
Work as a team, to agree the starting case, pull in different ideas and expertise, and agree what CI system, version control and packaging will be used:
Put the Source Repository, Artifact Repository and automated testing in place. Use a fake (but life-like) production environment to complete the Deployment Pipeline. ?
This may take a couple of weeks, and will feel like a lot of work and a significant up-front investment of time, but once up and running, the Deployment Pipeline will accelerate your progress, along with the team’s expertise and productivity.?
领英推荐
In my book “Continuous Delivery Pipelines ” I go into all these steps in more detail, and I have a FREE webinar on my training site which could be helpful if you are just getting started.
Next Steps
Once your Deployment Pipeline is up and running, your next story will only need tests and the code!?
Over time, add more checks and more sophistication to the pipeline: other tests and whatever else is necessary to determine the releasability of your code, e.g security, performance, resilience, data-migration, etc., depending on the nature and complexity of the software being developed.
And, sooner rather than later, automate version-control and Infrastructure as Code for the Deployment Pipeline itself!
To Speed up your Deployment Pipeline:
Why Do You Need a Deployment Pipeline?
A Deployment Pipeline is a falsification mechanism, where you can fix, and learn from, failed tests quickly. It is a platform which enables the measurement of test results, produces data about cycle-time, stability and throughput, and which we can use to make evidence-based decisions.?When new code has completed the transit through the pipeline, there is no more work to do and the software can be safely released into production.
The Deployment Pipeline is NOT:
If you would like to get the most out of your Deployment Pipeline and get all the benefits of:
... take a look at my Advanced Training Course: “Anatomy of a Deployment Pipeline ”
There’s lots more advice and discussion about Continuous Delivery, Deployment Pipelines, Automated Testing and other Software Engineering topics on my CD on YouTube channel.?Take a look and subscribe if you find my videos helpful.
Data Analyst | Data Fanatic | Generalist
2 年Great post Dave Farley! While a lot of this is still outside of my wheelhouse, it was really informative ??
IT Business Analyst for ABN AMRO Bank NV
2 年What are your thoughts when it comes to mainframe systems. How do we implement automatic (CI CD) testing where most of the process are in JCL batches and screens are in CICS / Telon.