Mastering The Dark Arts: A Guide To Predictive Operations

They say data is a magical thing. The silver bullet to all our problems. The mystic crystal ball. We harvest it and transport it (data engineering). We draw nice pictures with it (data analytics). And of course, perhaps most lucrative of all, we weaponise it and ask it to tell us what the future looks like (data science, prediction in particular). However, the fact remains that whilst analytics are becoming more and more commonplace within the aviation operations landscape, applications of prediction and predictive modelling are still exceedingly rare.?

So how do we take the next step and become masters in the dark arts of prediction and unlock the massive potential applications it can bring us?

Demystify

“How does this thing work?”

Prediction and predictive modelling is still a fairly new phenomenon in our industry. Change is scary. So it’s important here to build trust and comfort with the user so they are truly comfortable with what a predictive tool can and cannot do. It’s important to consider how we can address questions such as:?

·?????What is this predicting?

·?????How is this being predicted? What are the variables/features being used?

·?????What is the accuracy like?

Having supplementary analytics to accompany a model is great here. Whilst it is extremely difficult to pinpoint a particular change in a variable/feature towards why a model output has changed, supplementary analytics help to provide some much needed context. For example in a model that predicts turnaround time, stats on passengers, baggage and cargo are useful in helping the user rationalise the output. However, care must be taken to not overwhelm the user with information. Make it snappy, punchy and visual.

Design

“What is this showing me?”

?The users understand how our model works (kind of). Great. Now how do we go about maximising the impact of our model operationally? This is challenging given the novelty of predictive tools. We are often exploring entirely new use cases to the business which are unexplored frontiers. It is key here at least in the initial stages not to get bogged down in the specifics too much. Instead focus on:

·?????Small, testable iterations

·?????Simple and understandable outputs

·?????Limited use cases

We must accept in the initial stages that model outputs may not be fully useful to the user. This does not mean that the model is useless, but instead will unlock key user insights that will make it easier to make the model and its outputs more relevant to the user. Small, testable and scalable iterations are key.

Decide

“What if it’s wrong?”

?We are an industry that likes certainty. Years of prioritising safety and shunning uncertainty has made us risk averse creatures. As a result, we are often conditioned to waiting until the last possible minute to make a decision just in case an event does not happen, or if a window of opportunity opens to mitigate the effects. Most of the time unfortunately, that opportunity never comes and we end up delivering a terrible customer experience.?

This phenomenon is referred to as outcome bias. In a nutshell, it is where decisions are made with an overemphasis on achieving the best hypothetical (sometimes fantastical) outcome rather than making the best decision with the information available at the time. This mindset defeats the purpose of having any sort of predictive tool and is perhaps the most difficult thing to overcome when implementing a predictive tool. It is important here to:

·?????Make teams aware of their outcome bias

·?????Create a safe environment where decisions made in the operational window with a suboptimal end outcome are not punished

·?????Build a culture and KPI framework around making the best decision with information in hand, rather than on the end goal

The human element here is perhaps the most overlooked but complex component. Again, it is important here to build trust and ensure that a decision making framework is in place that is tailored around making the best decision with the information available at the time.

TLDR

·?????Be transparent about what your predictive model can and cannot do. Make the outputs simple, and build trust

·?????Work in small, testable iterations. It’s fine if your predictive tool does not see full usage immediately

·?????Build a culture and framework that rewards sound decision making based on information on hand, not one that incentives outcome bias