Continuous Integration, Delivery & Deploy - It's part of our lives !

Stakeholder: What do you mean by 2 more weeks? It was almost done yesterday wasn't it? I was waiting for it for the past 2 weeks and now it will be almost a month. Can't you do anything about it?

Product Owner: Yes it was but we found a bug which we can't fix and test in time. Our 3rd party provider never mentioned that X has a dependency on Y. It will be ready tomorrow but have missed today’s release deadline.

Scrum Master: Agree, I have managed to get to the bottom of this but unable to mitigate as contractually they are within their SLA. Look at the bright side though, we have managed to push through 6 other items as planned. The PO and the members have done their best they could.

Stakeholder: I understand. It's time we do something about providing a solution to these last minute issues. It’s practically blocking a high revenue business critical feature.

Product Owner: That's true.

Scrum Master: We can try continuous delivery as an option.

We have heard similar complaints several times (at least I did). If you are part of a development team at any level, you probably know what Integration, Delivery and Deploy means. If not then you have this post to guide you to understand the basics. Before we dive deeper, it is important to state a fact -

The terms 'Integration', 'Delivery' and 'Deploy' are not appealing and worthless at times - if it doesn't have a "Continuous" flow.

If they are still appealing to you, you are probably working within a team who prefers traditional approaches like waterfall over the 21st century approaches to become agile or lean or anything that promotes continuous improvement and based on Empiricism.

Continuous is, in fact the most underrated word in a product development life cycle. If you use it often at your work, pat your back and join me in spreading the word. This article is targeted towards non-technical professionals working around a development team, ideally a stakeholder/product owner/alike. It can be a good recap for those who don’t feel 100% confident about the topic as well.

Feedback - now, this is very appealing. It is quite powerful on it's own. Imagine what it represents when we talk about "Continuous Feedback". Feedback in the context of software development means reviewing deliverables after the Implementation + Integration + Delivery (+ Deploy, if we are capable). The moment we go deep inside a "Software" discussion, we end up taking about coding, testing, processes, tools and practices to establish this feedback cycle. This is where many stakeholders loses interest as they don't give a damn about how we make it happen. Technical experts are hired to help translate those low level information in to high level business goals, not the opposite. Stakeholders just need what they have requested, preferably on or before the timeline proposed.

Definitions

Continuous Integration (CI)

"Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible." - Martin Fowler

Continuous Delivery (CD)

"Continuous Delivery is a software development discipline where you build software in such a way that the software can be released to production at any time." - Martin Fowler, again.

Continuous Deploy (CD - yes I know)

Simply, the act of making a completed product available on Production or Live environment, continuously. - well, me.

It is not the same as continuous delivery. It is confusing because the acronym CD is used for both. Delivery can be on a UAT or a Staging or a Demo environment, which although resembles the production environment to some extent, may not really the same. Continuous Deploy, hence, is not recommended unless we are extremely confident.

The power of "Continuous" (Stay with me)

If you are a heavy commuter like me, live in South of England, use overground DLR/Southern Railway/Thameslink or similar plus the London underground, you are already witnessing the "Feedback" loop every bloody day. The difference is - they all represent a unique feedback cycle which may or may not suit us.

Unless you have given up and made peace with it, you probably feel a shift on your stress level when travelling on different services. I am going to use an example journey from my home city to London St Pancras to explain this.. hold my beer.

Continuous Commitment and Integration

What wakes me up everyday is the joy of working as an Agilist and not the excruciating pain of travelling to London, everyday. Travelling becomes a default daily commitment for a greater good. To commit every morning without a fail and to reach the train door in time I have to make sure that I leave home on time, everyday.

To repeat this everyday, as a human, I have to make sure that I play an hour of World of Tanks (yes, everyday) and get enough sleep. Unfortunately we humans also need a “turn it off and on again” the way our PCs do, unless we tend to screw up badly. This is non-negotiable and if we are not doing what resets our day, we ARE going to fuck up. It's a matter of being committed "continuously" towards what we have to do and make it our second nature.

When we compare this to CI in software development, we can see where we may be falling behind (if we are). If we are not committing everyday on our codebase and merging the latest changes (reset), chances are we will always see side effects that waste more of our valuable time. We may be working on a branch which should have been synchronised days ago. Hence. everyday. To ensure a healthy codebase, therefore, we need tests to run continuously whenever we commit for the fastest feedback on the changes made. If it breaks anything it's smarter to fix it now, rather than blaming the lack of sleep later.

Delivery and Deploy - How Continuous are we talking about?

As much as we are capable of, as fast as we can. It's a culture thing and very much depends on our capability and the people around us. It depends on budget, mindset, commitment and necessity. It can be a week, few days or few times a day which varies widely with the type of product and any dependencies around it. It depends on third party teams we are working with, clients we are providing value to or simply the reason that we are trying to improve "continuously" aka Kaizen.

Whatever the reason, it is always preferred to choose the fastest option and use the right approach to get it delivered without waiting for a stage gated process of batch deployment, treating it like a ceremony. If we are making a big deal of it, we are already accepting that we need improvements.

Back to travelling…

Choose what works for you over what is "Best Practice"

After I reach the Brighton station, I see 2 train services which varies way too much in terms of providing value to me as a commuter. T (Thameslink), stops in almost every station in the route, as it provides a "cheap" service and takes more than 1.5hrs to reach London St. Pancras. G (Gatwick Express, same owner), stops in limited stops and reaches London Victoria within an hour or so; and yes it is expensive. Around 20% more towards the cost. T services are reputed for being late in a regular basis and can be quite frustrating at times.

Careful observers might have spotted that I mentioned G only reaches to Victoria. Which means to reach St Pancras it will take another 30min anyway. Technically, I am not saving any time and literally spending more money. But the “Values” I am receiving are more important to me such as, a peaceful travel where I get a seat which faces the way the train is heading (yeah.. you are not the only one !), free wifi (most days), socket for charging, table for the laptop and always be on time. From Victoria, the tube is also extremely reliable even though it IS a dependency.

In software delivery, this is what matters the most. True, 2-3 commitment of continuous delivery a week is possible and is the end goal for some but is it possible with your skill set, dependencies or culture? May be not. If you are being forced to do so, you will most probably go and look for better options in your career elsewhere. We can aim for it but we shouldn't try to be the superman we are not, we may actually won't even need it. So technically it will become a waste of time, effort and will create a stressful environment.

Stress is a bitch !

Do you experience a shift on your stress level when travelling via Overground vs Underground?

The T service have a train every 30mins on peak hours to St Pancras. The G service have it every 15mins giving me a flexible and less stressful commute, on top of what other benefits it is providing.

Even though it is stage gated, it just works for me.

There is another reason. If we manage to reach any London terminal and see a Tube Strike.. no brainer, there are other means to reach your destination. It will delay your journey for that day but not everyday.

Even more stress free flexibility - The underground Victoria line have a service every minute or two. You can practically watch a train leave and still don't give a damn; the next is moments away, as close as 100m. That's the power of "Continuous" - anything.

As a Stakeholder..

Similarly, a stakeholder should be stress free about a missed release and shouldn't worry about missing a delivery window; just to be told that they will have to wait for another 2 weeks even though it will be ready for production the next day. That's the most irritating "process" which doesn't makes sense to them and to anyone who favours common sense.

How do we handle this stress as a stakeholder? We do understand that things go wrong, after all we are professionals. It doesn't mean that we now have to wait for another 2 weeks. So we will stress out and pressure the team to do a "Hotfix". Even though we know it can have bad implications, we will be biased to push it anyway and most probably will get what we wanted. We wouldn't care about another stakeholder as it is not important anymore. Our success is based on how well we get things out of this team, so screw the team or any other stakeholder.

If the team was delivering continuously, a deploy to production would be considered as just another step to add in the Definition of Done .

It’s a win for sure but for a short term. The fix we pushed out may have bugs as the development team members were rushed and may miss out few bugs. It will turn on us the moment we show our back. The reactions to this will be fatal. Now the team knows that the stakeholder is being unreasonable. They want to help but they can’t as the process don’t allow them. So next time when time will come to commit to something, they will commit to less amount of work to avoid a conflict.

Do we need this shit? It’s not worth the hassle trust me.

Aim to reduce batch sizes, continuously

If we are moving from traditional to modern approaches, batch release makes sense. In fact you cannot have an agile transformation without it. The fact that we have to reduce the feedback cycle from quarter/year to 2/4 weeks (as Scrum suggests for example), is still alien to many companies. ‘Continuous’ in this case is the length of the iteration cycle.

But this doesn’t mean once we have achieved the 2-4 weeks cycle, we cannot aim for even more granular continuous releases. The whole point of reducing batch sizes to as small as one item per release, is to provide support to the stakeholders’ need.

As a Development Team Member..

We don’t need a hostile environment, instead simply use common sense and be human for a moment while thinking about providing a feedback, if there is a need. Do a hotfix if that’s what it is necessary, just don’t say we can’t do anything for another 2 weeks. What will go wrong? It will slow the team down for the very same reason the resistance is there. At least we can reflect there is a change needed on process or tool or whatever that is blocking us. Stakeholders will be forced to understand as they are on the same boat and will stop pushing as it will force them to think long term.

Hope you all have a great week ahead, be continuous.. stay agile.. think lean ;)