Question #1 - What does data driven decision making actually mean?

It's commonly understood that decision-making should be more data-driven. At first glance, this seems logical: use data to inform future decisions.

Yet, upon closer examination, it becomes apparent that applying past data to future decisions is not as straightforward as it may appear.

Roger Martin eloquently stated on this matter: "The overwhelming majority of the time in business, the future is different from the past. Any inference that you draw from data analysis, which is about the past, is fundamentally flawed."

Given that analytics-based inferences are prone to flaws, what then is the solution? I consider three factors:

1. The Gawande List

2. The Fallacy of Averages

3. Benchmarks Invite Average Performance

The Gawande List originates from Atul Gawande's book, "The Checklist Manifesto." I have adapted his ideas to compile a list of variables that should be factored into every analysis. Here's an example:

Each of these variables is analyzed in every study to identify differences from the past and to forecast changes for the next campaign.

The fallacy of averages can also counteract an excessive dependence on historical data for future decisions. Averages can be deceptive, potentially skewing the narrative of market dynamics. An alternative is to segment the data as demonstrated below. By examining the outliers, new insights into customer behavior can be uncovered. For instance, in the first segment, two hundred customers showed activity levels above 41, which is well over the average. What sets them apart from the others, and from the 40,000 who showed no activity?

Another method to prevent being deceived by historical data is to avoid setting benchmarks as objectives. Charlie Munger aptly observed, "Mimicking the herd invites regression to the mean." This implies that benchmarks, which represent the average, can drive a marketer towards mediocrity. It's better to establish campaign goals that reflect the specific measures of success for the brand in question.

Next up tomorrow is: How precise does measurement need to be?