We need a recommendation engine for recommendation engines!

Analytics used to be so much simpler. You could go a long way with average response rates and logistic regression models. Using rule-based segments and model deciles was a big help in making better business decisions, and pretty much anyone could benefit.

The introduction of more complex modelling techniques, perhaps using big data, or in the cloud, with or without opensource, significantly increased the complexity of the challenge. However, it did not really change the core need: to understand the probability of ‘someone doing (or not doing) something’. This then drives the need to take other actions in response. If in the past we were hammering nails, sometimes the only difference now is that the nail is bigger….so we need bigger hammers.

When it comes to “recommendations”, however, business problems range from relatively simple to incredibly complex. The potential analytics solutions are therefore also more varied. There is a real danger of over-complicating solutions. This leads to higher cost, or slower speed to market. At the same time, however, it is also possible to develop an over-simple solution that misses revenue opportunities. Finding the right balance is hard.

I once worked on a business problem in telecommunications, on bundle recommendations. I calculated there were more possible offer combinations than there were people on the planet. I do not think that this was particularly unusual, either. This large combination of options means that working through them all, and making the right recommendation, was a big issue. Choosing an efficient recommendation methodology is the real challenge here - and this should be driven by business need.

Categorising recommendations and recommendations systems

I like to categorise recommendations across four dimensions, or perhaps questions:

1.      Who – i.e. the target group or person?

·        E.g. a specific customer or customer behaviour or segment of customer, or type of customer, or their basket, or their current product holdings.

2.      What – i.e what is being recommended?

·        E.g. a product, or an offer, or a bundle, or a category, or an action

3.      How – i.e. the information or analytics used to target the ‘what’ to the ‘who’?

·        E.g. their propensity to do something, or their similarity to other customers, or their preference for certain things

4.      Why – i.e. what is the business outcome to be managed?

·        E.g. increase sales, revenue, profit, customer satisfaction, mark down revenue

Remember too that there could be some complex analytics occurring in each one of these dimensions – for example a segmentation model across the group of customers – or a predictive model for every product, or linear programming (optimisation).

This is why I would classify these three (and many more) as recommendation engines:

·        Arbitration - picking the highest score across a set of offers

·        Collaborative Filtering - people similar to you, liked Film ‘xyz’

·        Factorisation Machines - perform well across a high volume of offers with sparse responses.

A recommendation engine for recommendation engines?

All these are valid recommendation systems. However, they all address the four dimensions in different ways, sometimes the dimensions may not even be obvious within the engine. Importantly, the business (and operational) performance will be significantly different across different problems with different engines.  

In summary, there is an enormous variety of ways in which the four dimensions can be combined, and that leads me to my question: Do we need a recommendation system for recommendation systems?

Perhaps that would be taking things a little too far. Selecting well, however, is crucial.

I can guarantee one thing. You or your business need to work through, challenge and clarify the four dimensions of recommendations. You really need to know what you need across each one. If you do not, the probability that you will deploy the optimal recommendation system will be significantly closer to zero than to one.