High levels of automation can also have negative consequences

I am a firm believer, like many of you, that well designed and calibrated automation augments human performance and avoids costs.

Automation is certainly not all or none, but can vary across a continuum of levels. I am sure that most of you want to reach a very high level of automation. However, the level of automation that you want is not always what is right for you.

You should be very conscious that high level of automation can lead to undesirable consequences. Don’t get me wrong, I am not saying that a high level of automation is bad and you should not aim for it. I am just saying that you should ensure that appropriate responses are provided in order to avoid unpleasant surprises.

The main purpose of this post is to briefly highlight?5 risk areas?that you should consider when deploying high levels of automation.

Mental workload

The evidence suggests that well designed and deployed automation can change human mental workload for the better. No more alienating manual tasks and more focus on value added activities.

However, automation does not always result in balanced workload. There are a lot of instances of “clumsy” automation where human workload has actually increased. Automation might be difficult to initiate, engage or monitor. These situations can simply increase both cognitive workload and if extensive data entry is required, the physical workload of the human.

If human interaction with automation is not simple, you simply run the risk of increasing human workload.

Awareness

If automation is highly reliable, then it may gradually reduce the human’s awareness of the procedures/processes and utilisation of underlying transactional systems. Humans just tend to be less aware when changes are under the control of automation than when they make the changes themselves.

So if the activities being automated are critical (from a financial, reputational, regulatory, etc. perspectives), organisations need to ensure that humans keep somehow an acceptable level of procedural and system awareness.

Take an example where automation has been enabled to run an activity with a lot of processing and validation steps, across multiple systems and dealing with high volume of data. This activity was manually executed by 3 teams before the introduction of automation. If a failure occurs, someone (an individual or a group) will need to manually perform all the steps following the existing standard operating procedures. There will come a time when humans will not be as skilled in performing this activity (codified knowledge will exist but humans will not be properly trained).

If awareness is somehow lost over time, the recovery of operations will not be a walk in the park.

Complacency

If automation is highly reliable, then humans may not monitor the automation and its data sources and hence fail to detect the occasional times when the automation fails. This effect of over-trust or “complacency” is greatest when human are engaged in multiple tasks.

Imagine a given automated activity is (almost) always executed without errors. You can be sure that during a certain period, the human will perform very strict verification checks but will slowly stop doing it with the same rigour (given that the automation is pretty much always successful). However, a successful automation run does not always mean a complete and accurate one !

So keep monitoring automation (continuously or periodically) and don’t let automation give you a false sense of confidence that things are always working well.

Reliability

Most of the benefits of automation are unlikely to hold if the automation is unreliable. Hence ensuring high reliability is the most critical task when applying automation. Automation reliability is an important determinant of human use of automation because of its influence on human trust. Unreliability simply lowers human trust and can therefore totally undermine potential performance benefits of the automation.

Take an example where automation is consistency failing, forcing humans to spend time managing issues, re-testing code changes, recovering operations and at the same time performing other activities that have been added to their daily jobs (thinking that automation will allow them to do more exciting things).

Ensure that automation you are deploying is very reliable. If this not the case, you are in serious trouble.

Costs of outcome

What if a critical activity has been automated and cannot be performed in a timely manner leading to significant regulatory fines or processing errors?

The risk associated with an automation outcome can be defined as the cost of an error multiplied by the probability of that error. So what would be your last call if you know that the cost of error is extremely high but the probability is low? Should you still go for high level automation?

For decisions involving relatively little risk, therefore, out-of-the-loop problems are unlikely to have much impact, even if there is a complete automation failure. Such decisions are usually very strong candidates for high level automation.

Ensure that the cost of negative outcomes is somehow aligned with your risk appetite. Can you actually afford to take such a risk?

Closing note

When running automation initiatives, organisations are somehow inclined to only see a bright future with a wide range of benefits. While optimism is great, it is very important to (at least) understand the potential negative consequences of automation so that you can re-calibrate automation levels (and ultimately balance automation desires with diligence) or mitigate the existing inherent risks (to reach an acceptable residual risk level).

But you simply cannot ignore them!

I would be very interested in hearing your thoughts.

If you would like to receive my future posts then please follow me.

This post has been inspired by an amazing research “A Model for Types and Levels of Human Interaction with Automation” published 18 years ago by Raja Parasuraman, Thomas B. Sheridan and Christopher D. Wickens. Content has been reused for the purpose of this post.