Deciding how frequently to deploy

I was talking with a colleague last week about whether they should increase or decrease their deploy frequency. They were worried that deploying more frequently could cause more bugs. They also were concerned because it takes time and effort to do a deploy.

There is a well-known optimization curve in queueing theory that talks about the optimal batch size. You need to balance the holding cost - the cost of holding off delivering a batch with the transaction cost - the cost of processing a batch.

A deployment is a batch of code you are shipping.

Our code does not take up space in a warehouse, but there are still a number of significant elements to our holding costs (this is also called the cost of delay):

• The cost of uncaptured value - when software isn't shipped, it's not returning any value. The longer you wait, the more money you're losing by not getting the benefits. It's also possible that the longer you wait, the less the actual benefit will be because the market is shifting or moving.
• The cost of delayed feedback - when you wait a long time to ship, you're waiting to get feedback. This almost always means that the cost of rework is higher once you realize that what you shipped isn't having the impact you expected it to have
• The cost to quality - the larger the batch size, the harder it is to identify the root cause when a bug occurs. This can impact customers if you aren't able to fix the problem in production, but even if you can roll back quickly, it's a ton more work for your team to figure out what went wrong an why. I love what Adrian Cockroft from AWS once said at a QCon session I went to: "when your code has a lot of bugs in production, you should be deploying more frequently."
• The impact to morale - the longer and slower it is to get code out, the less engaged your team is - they just don't get the sense that they are having an impact. Also, nobody enjoys manually shepherding out a deploy rather than slinging code and getting stuff done.

Our transaction cost is about how hard is it it deploy our software. Do we need manual approvals? Do we need to manually evaluate flaky test failures? Does it take an inordinate amount of time to run all our tests? Do we need to set up our environments?

If the cost of doing a deploy is high, you'll need a larger batch size - you'll be forced to have fewer deploys with more code in them.

This is why we need to be so careful about being too careful. Approval gates and slow or manual tests will force you to have larger batch sizes with all their resulting holding costs, including costs to quality.

This is why I always encourage teams to aggressively go after the things that are making their deploys expensive.

Build automated pipelines. Eliminate manual approvals. Reduce the number of flaky and slow tests using techniques such as testing in isolation, testing in parallel, having high priority tests vs additional tests you can run less frequently, and so on. Make your PRs smaller and easier to review and roll out.

Have this audacious goal: you check in some code, it goes to production, and it's a non-event.

Not only is coding more fun that way, but you'll reduce those holding costs and deliver significantly more value for your customers and your business.