How to help your team deliver continuously

“Continuous delivery” (or more popularly, continuous integration and delivery, or CI/CD) reached buzzword status somewhere between 2015 and 2017, and continues to be one of those things developers love to work on, and therefore also one of those things companies love to tout in recruitment ads.

But what it exactly is, and why a business needs it, are questions that tend to get drowned out in the din of excitement about tooling and practices. It is not impossible to find young “CI/CD enthusiasts” who cannot quite explain continuous delivery at a fundamental level, except in terms of tooling (e.g. Jenkins, GitHub Actions, CircleCI). As a result, you, as a manager, may struggle to understand the business benefit of investing time and money in these new tools and practices.

In such murky waters, it pays to start from first principles:

Requirements can never be fully accurate. Those who write requirements (clients or product managers) do not have software emulators in their heads to run and verify their written requirements up front. So requirements will (and must) always change after stakeholders see a first version of a built product. This is a reality engineers must accept. The best they can do is to demonstrate what they have built as early as possible. The longer it takes to do a demo, the more likely that time may have been spent developing the wrong thing. This is in fact the greatest insight of the Agile Manifesto (principle #3: “Deliver working software frequently”).

You cannot demonstrate back-ends, APIs or databases (unless your stakeholders are themselves engineers). So the traditional approach of building a system as horizontal slices (first the database, then the back-end and finally the front-end on top of it) does not work. Instead, you must build the system as vertical slices. Each slice includes the relevant database, front-end and back-end changes, and can be demonstrated to stakeholders on a regular basis.

However, engineers are often reluctant to release their work in small bits, because the process of releasing is often more cumbersome and frustrating to an engineer than actually writing the code.

Consider this scenario: an engineer makes a 2-line code change to a validation. It takes them all of 5 minutes to change the code, change the unit tests and verify that the change works on their local machine. However, the process of getting that change into production (or even into a QA environment) will look more like this: submit a code commit or pull request to the source code repository; wait for another engineer to review and approve the change; merge the change to the main code branch and wait for all automated tests to run (if there are any); notify the engineer in charge of deployments that a new version of a system component (in this case, a back-end), needs to be released; that engineer will in turn wait for several more changes to be submitted during the day; then they will build a new version of the component and run a deployment script, which will usually shut down the current component in the target environment, transfer a file/files representing the new component into a server in the same environment, update any configurations and restart the component; then some verification test will be run, either manually or as an automated test. Here, delivering the software took perhaps ten times more effort than actually writing the code. So engineers will naturally tend to batch their changes together and release infrequently to avoid this pain. It is this infrequent release habit that leads to more rework, as we discussed above.

Now consider an alternative: as soon as the 2-line code change is reviewed and approved, the engineer simply merges their code into the main branch. This triggers and entire sequence of automated tasks that end with a newly built component being deployed and started in the target environment. Ideally, a set of automated tests will also get triggered, verifying that the newly released component has not broken anything in the target environment. If this process can take place in minutes (rather than hours or even days, as it was with the manual scenario), your team can now be encouraged to release even small bits of working code without additional overhead.

This is the entire purpose of continuous delivery: build software incrementally and conduct demos to stakeholders as frequently as possible. It is the final stage of an agile software development lifecycle.

And yet, we often see supposedly agile teams releasing their work in large, infrequent chunks, even as the tooling is in place. This is usually a consequence of them chasing false efficiencies. Your first job as a manager is to make the team understand that the inefficiency of requirement changes and rework is far more insidious than any inefficiency involved in doing frequent build/test/deploy cycles. To this, you will often hear the response that rework could be avoided if only stakeholders could provide accurate requirements the first time around. This where you need to convince them that there can never be such a thing as accurate requirements first time around, because (as we discussed earlier) no human being has a software emulator running in their heads to verify a written requirement before actually seeing it in action. So rework is a must. The best you can do is demo early so that the stakeholder can change their mind early and limit the amount of rework.

Once the team abandons the expectation of “accurate” requirements, they can focus on the real problem: reducing the time and manual toils spent on testing and deploying their work. If the team still falls back to old habits of batching their work into infrequent releases, consider giving them this guidance:

The moment you have a working piece of code (i.e. tests are passing), STOP. Do NOT continue to write more code. Instead, release your code. At a minimum, make sure it lands in a QA environment. Only then should you start writing more code.

In summary, encourage your team to abandon the expectation of accurate requirements, avoid batching work into infrequent releases and relentlessly improve the speed at which the automated test/build/deploy cycle runs.