What can go wrong... and what will you do about it?

Sometimes, despite everyone’s best intentions, things go wrong. Good risk management can help ensure this doesn’t have an impact on customers but can still lead to wasted time and effort. Sometimes this is due to errors in the data feed, sometimes issues with deployment and sometimes because of miscommunications about what the model does compared to what was needed. Often in these situations it is important to hold a ‘post-mortem’ for everyone involved to get together and discuss how this happened and what we can do to avoid it happening in the future.

More and more I am in favour of holding a ‘pre-mortem’. Bringing together all stakeholders before any big model project to think through ‘what could go wrong?’. By doing this earlier in the process there is (hopefully) less emotion and finger-pointing and should help ensure everyone understands the risks and trade-offs with the project.

In these sessions I find it really important to focus on the risks by articulating them as ‘there is a risk that …, caused by …, leading to …’. This helps to avoid vague, generic risks and focuses on what actually could go wrong. Once you have this list you can also articulate the likelihood of the risks happening and the impact they would have. By categorising them in such a way you can spend your time thinking through how you might detect or mitigate your top risks.

To summarise, a large part of building successful AI applications is to have a strong focus on effective risk management. The more of this discipline you bring to your project, the less time you will be spending in post-mortems for failed projects.

Questions to ask:

• Prior to starting a big piece of work have you identified all the stakeholders?
• Do you have an active list of risks?
• How do you ensure your time is spent focusing on mitigating the most impactful risks?
• Who is responsible for accepting the remaining risks? Do they know this?

Last week: but is it working?

Next week: help out your future self