Occam’s Razor

Buying a new car can be a pretty daunting experience unless you know exactly what you want. Deciding on a make and model is just the start - at this point you are presented with a huge array of features, add-ons and extras. It can be very tempting to nod along and end up with everything without thinking through the cost.?

There are clear parallels for me when I think about the algorithmic choices available with Machine Learning. There are always newer, flashier algorithms, or ways to add more complexity to your current approach. Some of these choices will help you thrive.

Historically in our UK business logistic regression was the core algorithm which underpinned our critical underwriting models, the ones that helped us make key business decisions. There was a good level of understanding with the approach however a great data scientist never rests on their laurels and over the past 5 years the team has broadened their skillset to bring in more complex Machine Learning algorithms to support this key decision.?

At each step of the way the important thing to balance is increased complexity versus the improvement in model predictive power. Occam’s Razor is a heuristic focusing on parsimony or simplification. The philosophy advocates that when you are presented with competing hypotheses or models giving similar predictions, you should select the solution with the fewest assumptions. In a data science application this means being intentional about complexity you introduce through data or algorithms. Does this complexity lead to meaningful changes in your decisions or prediction levels? If not, then you should strip it all away.

Occam’s Razor remains more relevant than ever and an effective, pragmatic data scientist will always be looking to balance simplifying and improve model performance.

Questions to ask:

• If you are using a new technique: can you articulate the increased complexity in terms of risks?
• If so; do you understand the incremental benefits?
• Have you made an intentional (& documented) decision to weigh up these two dimensions in making your decision?
• How are you embedding an ongoing process to challenge and remove unnecessary complexity?

Last week: opening the watch

Next week: the jets and the sharks