BigQuery: Chapter 1B - Intro Continued

This time we are going to talk about the real reasons why we, as marketers, need to use data analytics, that too with the help of a tool as sophisticated and robust as BigQuery. In this chapter we will talk about the following:

1. Proposed Architecture
2. Introduction to Machine Learning
3. Sample projects

Just a reminder, if you haven’t been following the series, please start reading from below links, it would make lot more sense:

• Power of Statistics
• Introduction to BQ Series
• BQ Chapter 1A

And now that you have gone through these links, you are much better placed to read further, so carry on.

1) Proposed Architecture:

To conduct any advanced analytics, we need to ensure that our clean, accurate and complete dataset is residing in one single location. This means, data coming from your web analytics tools, your CRM, other 2nd and 3rd party sources should be residing inside BigQuery platform. Keeping in mind that all this data might have different fields, attributions and names, so we need to come up with a plan to create 1 seamless dataset of all that data is coming from these disparate sources.

Your model architecture should be close to the diagram below:

If you see, in this universe a user can come on your platform through diverse channels. All the behavioral data of your visitors is getting tracked within your webanalytics tools, and at the same time in your CRM databases. Now data from your CRM databases, webanalytics tool, and other 3rd party sources along with your Google platform data should be merged and stored within BigQuery, either through direct integration or through Google Cloud Storage.

• Once data is present in BigQuery, it can be used for either:
• Solving classification problems through Logistic Regression
• Predict values of different events through Linear Regression

At the point of time next question should be, what kind of problems can Logistic Regression or Linear regression solve for marketers? But before I give you the answer, let’s dwell into section 2

2) Introduction to Machine Learning:

In last few years I have been surprised by the keen interest of almost everyone, resources that people are investing and still persistent confusion in this field. So, let’s try to address some of the key terms here, from our topic’s perspective.

a) Artificial Intelligence: It is an area of computer science that deals with creation of intelligence in machines which could mimic human and other animals’ intelligence. The idea over here is not just that machines can act as a human but can actually apply one of the most basic and yet most important aspect of animalistic behavior, that is cognition, a process of acquiring knowledge through reasoning, thinking and learning.

b) Machine Learning: The ideal definition of Machine learning is ‘Machine learning is an algorithmic approach to making repeated decisions with data’. Hence it is a part of Artificial intelligence, where a machine tries to learn statistically and progressively to improve its performance of understanding the outcome without being explicitly programmed for it. That is, when there are no if else statements for a computer program, but it can learn what the output is going to be when input data is specified.

c) Expert Systems: This is a close cousin of Artificial Intelligence, where a machine is used for decision making processes by putting exhaustive if-else cases. Every input goes through a series of tests, and depending upon the result of each test, either next test is given, or the final answer is provided. These systems were used earlier, and still are in many cases. They can give an illusion of artificial intelligence but can’t improve itself the way artificial intelligence algorithms can.

d) Logistic & Linear Regression: These are predictive techniques of Machine Learning, where we try to either predict the value of a categorical variable, that is classify the output as either 1 or 0, yes or no, Red or Green etc. or predict the score of variable, like average order value, revenue that would be made next month, users to be expected in next 3 months etc. In any of the regression algorithm we try to find the curve that best fits the data points that we have observed in our data, and to predict it tries to extrapolate the curve and come up with the data points that we might get in future for the specified variable.

e) Neural Networks/Deep Learning: Now these are utterly fancy terms that people love to use, but to give you a perspective, these are unsupervised Machine Learning algorithms where we are not really trying to predict a certain outcome, but are trying to uncover the hidden patterns from huge quantities of unstructured data, like understanding the topography of a specific region on the basis of photographs, Natural Language Processing to understand the mood of the customer based of tonality, expressions and modulation of voice. There have been many times I have heard people asking me why I used linear regression, why not Neural Networks, well, both techniques have different purposes.

Now that you understand these key terms, let’s understand in section 3 about how marketers can utilize the regression techniques to answer their own business questions.

3) Sample Projects:

There are quite a few business problems that can be addressed through these simple yet effective advanced analytics techniques, to give you some examples:

a) Churn Rate Analysis: To understand which customers we can lose in next 6 months from our books can be an important piece of information. Every customer saved is customer made, based on this notion, you can create this classification model to run on your existing customer base, and understand which customers have highest probability to churn.

Can be widely used in industries like: Newspaper Publications (for both, their online and offline subscribers), Retail, Banks, Schools/Colleges, Telecom.

Next Steps: Once you have identified the customers with highest probability to attrite, next step has to be to re-engage with them through campaigns, offers, discounts, if nothing else, then just a personalized touch. We have seen in the past how a simple ‘Hi how are you?’ call from one of the bigger regional banks of Australia drove up the sentiment, NPS and experience for their customers.

b) Retargeting Strategies: So, you have 1 million customers, with conversion rate of 0.8%, with return customers adding 70% of your orders, you have a need to retarget your customers. But then you can’t go after everyone. So, you apply logistic regression on your customer base to understand who has the maximum probability to convert depending upon their actions, referring channels, and days since last visit, you can come up with a list of top 10% customers that can provide you 50% revenue. Now rather than going after everyone, you will only focus your strategies for these 10% customers to drive your revenue uplift.

Can be widely used in industries like: Retail, CPG, Hotels, Airlines.

Next Steps: When you know a narrow group of customers who have higher probability of delivering bigger impact on your top line and bottom lines, it becomes as much easier for your media teams to customize and personalize messaging for these customers through Display, Social, and Programmatic Channels. Whereas this methodology will definitely help you bring the costs of your media down, it will also help you increase your conversion rate & revenue.

c) Customer Lifetime Value: Though no 2 customers are exactly the same, but we still can segment them on the basis of common attributes and then statistically calculate the current and predictive lifetime value of these customers or customer groups. This will help us to understand the low value customers and strategize to improve their lifetime value, and at the same time, it will help us identify high potential lifetime value customers, thereby helping us create strategies to take care of them through different media messaging.

Can be widely used in industries like: Telecom, Hotels, Airlines, Retail, Pharma & Wellness, CPG< Banking & Insurance.

Next Steps: Once you have a fairly good idea about the potential lifetime value of customers you can segment that 4 quadrants, Low-Intervention:Low (For least 25 percentile), High-Intervention:Low (For next 25 percentile), Low-Intervention:High (For next 25 percentile), and High-Intervention:High (For top 25 percentile). Needless to say that strategies of messaging for each of these segments would be different. Your media team can come up with internal campaigns and targets to push some of the customers from High-Intervention:Low to Low-Intervention:High etc.

d) Expected Marketing Value: We reach out to our customers through multiple channels, multiple mediums and platforms, but somehow, we don’t really measure the combined impact of these channels. Through traditional attribution algorithms we just attribute the impact to the most recent cannel/medium used, in full. Now a very scientific methodology indeed. A linear regression model can help us gauge the isolated as well as the combined impact of running different campaigns through different mediums and channels. Hence, we will not be measuring the impact of the campaign, but the whole marketing strategies taken together.

Can be widely used in industries like: Pretty much everywhere

Next Steps: You can determine the high impact campaigns and channels in your marketing channels, and those which are not having a desired impact. At the same time, you can determine which strategies are actually negatively impacting the final conversion numbers. You can create this model for different regions/markets and even customer segments, thereby determining the ideal marketing mix for a given set of customers or regions.

And this is where we will take a pause for this week. I know that this turned out to be little more than bite sized probably, but I have tried to keep it simple, so it shouldn’t be too difficult to understand the concepts, and where we are heading towards. Though I did try to give a few use cases, but by no means it’s an exhaustive list, like regression techniques have widespread usage in financial industry to determine credit risk, at the same time you can come up with many marketing use cases of these tools, and will really appreciate it if you can post some of those in the comments below.

In next installment we will be talking about the UI of BQ, how to initiate your first project, integrate it with Google Analytics, adding your own data sets, and merging some of the data together to get a better picture of your customer behavior on digital platforms. Let me know about your thoughts of how to refine it further, we can probably crowdsource this attempt of creating a wholesome working documentation on BQ.