Why the deploy-ability of your software is a craftsmanship value

My fourth "...ility" in my series of articles on non-functional aspects of delivering code into production, as part of the set of values that should be applied in a software craftsmanship environment. Craftspeople really do care about the deploy ability of the code and push for dev-ops members to be part of the team in your engineering function.

In the way back when, in the long-long ago, in the mists of time the word "dev-ops" was a fever dream of the future as yet unrealized. These days everybody seems to understand about dev-ops, some people even legitimately know what an SRE is these days, back when I started software development such a term was not a thing. What was realized and understood, however, was the need to build and deploy the software very easily, quickly and reliably. I wish to highlight what it will mean for your startup or scale-up if you get this right and what it will mean if you get it wrong.

When you get it right and you can deploy with ease, the following benefits can be realized:-

• Because you can deploy with ease - and yes you will so this the moment this becomes easy.... You will move towards that nirvana state of achieving CI/CD continuous integration, continuous deployment and reducing the overall risk on your project. I worked in one startup where we deployed straight into production every day for 30 days straight as part of our "A fix a day keeps the complaints at bay" process. It worked very well. If you do a deploy a day, the risk each day of a bad bug getting into the wild is very, very low and if you do...
• After deploy to production if a bug is discovered you can do a fix - often quickly as its the only change - and you can have it deployed and live 30 mins later - often before a good portion of your customers even notices.
• Your developers get to see their code in production, feel happier, get the feedback they need and are recognized for the difference they have made
• Customer service becomes happier as the customer base becomes aware that changes are frequent and if they have grumbles they know that a fix will be coming soon.
• Stakeholders in the business fight less over resource because everybody knows you are moving as fast as possible, and things happen quickly
• You can accelerate the product road map because you can make lots of small deploys over time, avoiding big - bang project deployments. I personally have never seen a big bang deployment work happily and smoothly.
• You can have multiple environments for demo, UAT, prelive, develop with ease showing different versions in different states with different features.
• Reduce risk of project failure through the reduction of change risk impact
• Increase the speed of the "build->measure->learn" loop so that learning is faster for the business
• Provide early validation of ideas, concepts and enable a pivot or persevere conversation to happen weeks or months sooner than otherwise possible
• Save operational costs in deployment time
• Happier customers, who see change happening on a regular basis which helps with "Customer collaboration" aspect of craftsmanship values

When you get it wrong and you can't deploy with ease, the following symptoms can arise:-

• Small bugs and glitches become a big issue, to the extent you will lose customers through a build up of annoyance over time.
• Developers have anxiety, and lets face it many of us are anxious enough anyway
• The act of doing a production deployment can involve multiple members of a team working weekends, late into the night. All with the associated costs and frustrations. The culture of engineering can fall apart if this happens too often.
• Developers can leave for a better environment to work in
• People/stakeholders blame each other, and waste time in order to "cover their ass." in stead of driving the product forward
• Risk with each week of software change compounds, and after a month begins exponentially to increase and after 3 months of changes the risk of severe large scale business affecting issues looms large. If its 6 months then you are in effect re-launching a new product.
• You may not be sure about the database as the codes behaviour with the tables and data state has changed multiple times, leading to potential loss of data and corruption of data integrity - giving rise to complex migration tasks which often have no way to recover from.
• The business doesn't learn quickly enough, isn't able to validate ideas and the costs associated with failure become prohibitive - even existential.

So for a software craftsperson what sort of things do I have to do to have my software deployable?

From multiple of the books I have consumed many people have said you need to have a 10 minute build, I would say in pragmatic reality I would revise this and state that you need to hit this as a benchmark:-

"Can you build, test and deploy your software into production in less than 30 mins from a single button click or script?"

This is then allowing up to 10mins for the build, up to 5mins for the unit tests, integration tests etc., and 15mins for apps into production/docker containers in your chosen cloud etc... I know that big boy companies with thousands of machines can't do this as quickly as because their infrastructure estate is too large, however - the sentiment is still there, It needs to be automated, it needs to be smooth, if you have to you can roll back if you need to, it reduces risk, it reduces cost and most importantly it enables you to respond to change - it just works. This is where you want to get to.

Make sure the 3rd party packages that your software relies upon are clearly defined i.e. version, location, with backup locations to fetch from. So that they can be fetched at build time and NOT placed in your repo as binaries to be checked out - I shudder when I find people who have done this. The reduction in size of the checkout and the performance gained is dramatic. Yes you can still pin the version number if you have dependency anxiety but its vital to keep your code separate from your 3rd party dlls.

Have your code as easily portable as possible. I.e. so that it can run on the widest variety of platforms as possible, Windows, Linux, Mac etc... so for the .Net people out there use .Net core NOT .Net framework, and if you are on .Net framework - please move on ASAP.

Make sure all secrets, keys, database connection strings, tokens are not hardcoded but are instead referred to from a separate file or location (there are lots of ways of doing this in fact an entire topic on its own). But it means that whatever CI/CD system you are using can replace the tokens at the time of deploy.

I.e. Build once - deploy many.

Design your unit tests to be able to run concurrently, don't have any tests that rely on the exit of a previous function to be "in the correct state". If you make sure that all of the tests are correctly mocked and asynchronous it wont matter if you have 7000 tests, you can still be running 100 or more at a time and your tests will complete in 5 mins or less. In my last project we had 7853 tests and they all ran in less than 3mins. This means you can find out before its checked in if all is ok, and it won't slow down the deployment process.

If say you have stateful react components and testing them asynchronously with cypress gives you failures, restructure the tests or restructure the components, or create a pipeline that will run isolated to remove this pain. Yes it can be hard to get your tests in order - but it will mean a big improvement on speed.

When building for cloud deployment building some of the containers can be a slow process (often far longer than the build itself), so you can pre-build an approved container and have it ready

There are many tricks that developers, dev-ops and SRE's can do to give you the speed of deployment that you need.

The attention that a software craftsperson will give to making sure that your software has "deployability" will go a long way towards powering your business and you will find that your engineering function drives the business and enables rather than costing and hampering.

Regards Julian