Next product to Buy

Introduction: how often if happens that you do an on-line transaction using the payment gateway for a famous bank and within minutes you get an offer consisting of exactly the same product. How did the bank manage to convert a customer transaction to a new X-sell lead? Definitely, streamlining of the process helped the banks take quick calls. But more importantly big data has helped banks to recognize each customer and identify his needs. If you are about to say bank use propensity of a customer to buy the new product to make X-sell offers, you know just a fraction of the whole story. These calculations have modified a lot in last 5-7 years. Today, banks perform an infinite number of calculations on a single customer to come up with the next best product for him/her. There are basically the following key ingredients in creating this near next to wow offering

Determining the Data Set

A bank is a bank right, I have been paying car loan to a different bank using my regular savings account from my primary but my primary never offered me a pitch to transfer over my car loan, well I think the fundamental problem is they don’t even know about it. In terms of data the car loan division does not talks to the banking division inside the same group, then there are departments inside a bank which can almost exist as independent organizations and this is just structured data. Creating an Offering like Next product to buy (NPTB) will actually involve social, machine and many other data sets. The journey to the end of the rainbow starts here, the trick is to start small, agile and launch NPTB in iterations. A most probable journey will be consist of the following broader steps.

Identify the key Structured Data Set

Purchase history is definitely a good start, in traditional market basket analysis a customer’s basket is calculated for few iterations and then there is a projection done for the next iteration. For NPTB multiple years of purchasing history needs to be clubbed together in the single big basket and then the algorithms need to be tuned for the projections? The other key data sets which will play a key rule for the NPTB algorithms are demographic data, product portfolio data, transaction data and contact center data.                                                                          :) Gosh! Didn't I already scared you enough!

Identify the key elements of unstructured data sets

A true Next product to Buy (NPTB) for financial services cannot be created without bringing in the social media, call center and app data. The key sources for unstructured data and the complexity beneath them should be well determined, also at this stage itself there should be fair enough idea of the integration mechanism (verbatim?), the platforms (MAPR is out, spark is in, what will be next?) and the modelling language (SAS is expensive, R can only model, Python can deliver but there is a new kid on the ground Julia)

Taming the Bull (the algorithm)

Finally the last piece of the puzzle lies in choosing the correct algorithm and tuning the algorithm but there is a bigger question which needs to be answered first. It’s hiring the best decision scientist who can choose this algorithm, scrape the data and then tune the algorithm. A data scientist who actually delivers the final code and does not just projects the models in R.  What I have learned from my guru and he in turn has learned from his guru is that the best data scientist should be a combination of a hacker, decent domain knowledge and statistical skills. Something like the following

Once our superman data scientist has arrived, its months of efforts playing around with scraping/munging data and tuning the Bayesian or the random Forests of the world.

Just my 5 cents on NPTB , We will talk more about NPTB with examples in my next blog……… Njoy