Automate (Most) Things

"There is a lot of automation that can happen that is not a replacement of humans but of mind-numbing behaviour." – Stewart Butterfield

Earlier this year, I was faced with a task that seemed mundane (I cannot remember the details, because the only important part is that I was not looking forward to it). So, I did the most rational thing: I procrastinated.

The deadline didn’t magically disappear; I chose to play the let’s-pretend-that-it’s-not-there game. I used my time to solve more complex problems and even to participate in an Innovation Challenge (ours was in the top three of over 250 ideas submitted). Another part of the reality was that I would repeat the task at some other point in time.

Time waits for no one, and eventually I had to face the music. So, I did. But I remember considering that if I needed to do this, I would find a way to avoid performing the task again in the future. Procrastinating repeatedly didn’t make sense. Passing it on to one of my colleagues did not make sense either.

This seemed to be a perfect opportunity to address the task with the grandeur of a software developer: write some code. I got myself a copy of the book Automate the boring stuff with Python, and quickly got to implementing a solution. By no means elegant, but it served the purpose. And with some refactoring afterward, I had myself a winner!

This brings me to today’s topic: Automation. Partly because of its role in the DevOps ecosystem, and partly because I detest inefficiencies.

Automation in DevOps

In a previous article, we explored the importance of a DevOps culture, and how it serves as the guiding light of any solid DevOps journey.

Automation is part of the technical foundation of DevOps; it streamlines the various development and deployment activities, allowing time for the teams to focus their time, energy, and creativity on more complex problems. It also creates time for the big differentiator: Innovation.

Have you heard the mantra: Automate Everything?

Everything.

…Everything?

Nope, not on my watch. Automation is not the silver bullet often sold to leadership. It entails careful consideration, planning, evaluation, and an intense level of problem solving.

This is how I normally approach the process of Automation:

1.   Identify the process to be automated:

There are so many integrated processes… where to begin? I start with mapping the software development lifecycle (from concept to cash to maintenance). The next step is to identify the tasks and activities that constrain the flow of value; the tasks that are repetitive by nature – not complex.

2. Understand the current process:

• What triggers the process?
• Which external factors / decisions influence the process?
• What data / factors serve as inputs to the process?
• What is the current outcome of the process?
• What is the expected outcome of the process?

3.   Classify the process / tasks:

A Harvard Business Review article highlighted the following continuums for classifying candidates for automation:

• Repetitive vs Variable Work: Is the work predictable or unpredictable?
• Independent vs Interactive Work: What degree of communication and collaboration with other people or processes would be required?
• Physical vs Mental Work: Is the work by nature mechanised and simple, or does it require a fair amount of cognitive engagement?

4.   Determine what we stand to gain from automating this process:

In my experience, we cannot (and should not) attempt to automate everything. One of the biggest factors to consider is what we stand to gain from automating a process.

• Time saved by no longer performing a manual task several times
• Faster process execution
• Reduced cost of the process (not taking up someone’s time)
• Predictability in terms of the outcome of the process
• Knowledge gained by understanding the process to be automated
• Opportunity for teams to perform more complex activities
• Happier employees (no need to procrastinate)

Once a manual process is considered a good candidate for automation, the implementation of such automation is a reward in itself. The elation of knowing that you (and no one else) would need to perform such a menial set of tasks again. It really brings me JOY!

Circling back to DevOps, the processes we usually automate include the following:

• Backlog management via an Application Lifecycle Management (ALM) tool. These tools simplify and support the process of centralising the visualisation of work items, automatically track metrics (lead time, cycle time, burndown charts, cumulative flow diagrams), and facilitates communication (via targeted alerts).
• Automated Build process that takes the source code create by developers, and consistently converts it into machine executable applications. This includes the process of running static code analysis on the source code to determine both the code quality and the security compliance. This is often referred to as Continuous Integration (CI).
• Automated testing, be it in the form of unit testing, integration testing, or even functional testing. Performance and load testing are often automated to make it easier to rerun these tests for subsequent deployment tasks.
• Artefact Management to store and version the binary files that were produced from the automated build.
• Continuous Deployment (CD) to automate the deployment of the produced binaries to the various target environments. Rich reporting on the successful deployment also makes it easier to manage versioning of software on the various environments.
• Monitoring and alerting reports on situations where predefined metric thresholds are reached (whether from an application, operating system, storage, or networking perspective).

"The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency." – Bill Gates