Avoiding the Data Causality Trap: Why AI Needs Your Business Savvy to Succeed

Imagine you're the head of sales for a fast-growing software company arming your team with a sleek new CRM promising ‘guaranteed conversion’. Powered by AI, it analyses your sales funnel and assigns win probability scores. Despite a nagging suspicion borne from years of experience you trust the AI's wisdom and unleash your team on the opportunities, showering them with sales pitches and proposals. Months pass, but the promised surge in deals never materializes. Conversion rates remain stubbornly unchanged. What went wrong? You trusted your CRM data, but it failed to tell the whole story.

This data causality trap can occur in any predictive use case, not just sales. Relying on siloed data, like your CRM system or website analytics, can lead to misguided decisions because it often fails to capture the bigger picture and the underlying factors that truly drive outcomes.

The CRM example highlights this perfectly. While it might tell you which companies clicked on your ads or downloaded your white papers, it doesn't reveal the complex web of external influences – budget constraints, competitor offerings, internal company politics – that ultimately determine whether a client signs on the dotted line. Correlation is not causation, and generating conversion predictions based on your CRM data is like using a compass with a hidden magnetic anomaly: it might point you in the general direction, but it could lead you wildly off course.

Remember: AI is only as smart as the data you feed it. It cannot ‘know’ or ‘suspect’ that there is a bigger picture or that it is being fed with wrong or biased data. Therefore, ensuring we apply the right data for the task at hand is a responsibility that cannot be delegated to AI. It is a fundamental aspect that needs to be assured well before we apply the expensive tools and resources of AI to the scene.?

Here is another popular example of how data can lead to wrong conclusions. During World War II, US military researchers faced a critical problem. Many bombers were getting shot down on runs over Germany. The researchers knew they needed hard data to solve this problem and went to work. After each mission, the bullet holes and damage from each bomber was painstakingly reviewed and recorded. The researchers poured over the data looking for vulnerabilities.?

The data showed a clear pattern. Most damage was to the wings and body of the plane. Their conclusion was - Increase the armor on the plane's wings and body, nevertheless bomber survival did not improve. So they approached Abraham Wald, a genius statistician at Columbia University for advice. Wald recognized the bias in analyzing the planes that survived instead of those that were shot down. To understand the vulnerabilities you need to analyze the damages to the bombers that were shot down, but these, of course, were not available. Therefore, Wald proposed that the military reinforce the areas where the returning aircraft were unscathed, inferring that planes hit in those areas were the ones most likely to be lost. This sort of biased data is known as the survivorship bias.?

So, how can you, the business-savvy executive, navigate this data maze and ensure your decisions are grounded in true understanding? Here's the good news: you don't need to be a data scientist to crack the causality code. It's about applying your business acumen and common sense, asking the right questions, playing devil’s advocate, challenging the team to find logical flaws and biases.

Ask yourself, before feeding your AI:

• What's the business goal? Predict churn, optimize marketing, or boost efficiency? Knowing the "why" guides your data hunt.
• Beyond internal data: How might market trends, competitors, or wider economics impact your predictions? Don't get siloed!
• Beware of bias: Is your data skewed by confirmation bias (seeking validation), survivorship bias (ignoring missing data), or anchoring bias (overly influenced by first findings)?
• Quantitative & qualitative: Is your data diverse? Go beyond numbers! Gather customer interviews, employee feedback, and industry reports for richer context.

By actively engaging with your data and asking these critical questions, you ensure your AI project will be built on solid foundations of relevant and complete data. Don't let biased or incomplete data blindfold your AI journey!

In conclusion, the allure of AI can be enticing, promising quick fixes and magical solutions. However, as we've seen, relying solely on AI without a healthy dose of business acumen and critical thinking can lead down a path of costly failures. By asking the right questions, challenging assumptions, and ensuring your data is relevant and unbiased, you can avoid the "data causality trap" and ensure your AI initiatives are rooted in reality, ultimately driving true business value. Remember, AI is a powerful tool, but it's up to you, the business-savvy leader, to guide it in the right direction.

In the next chapter of this series we'll delve into Supervised vs. Unsupervised Models: Understanding the difference and which best suits your business problem.

Feel free to comment below and share your experiences! Your contribution to the discussion and knowledge sharing is valuable.