What Data Cannot Tell You?

I had spent years working with data, trusting that the numbers would always tell the whole story. My dashboards were meticulous, and my reports were precise. If there were a problem, the data would reveal it, or so I believed. But one day, while working on a Telecom customer's data, a sudden, unexplained drop in revenue appeared on my screen. The numbers showed the dip, but they didn’t tell me why.

At first, I looked at the usual suspects: billing issues, customer churn, and network failures. Nothing stood out. The data was giving me nothing beyond the obvious. Frustrated, I expanded my search, reaching out to colleagues in the field. That’s when I learned the shocking truth: the Taliban had brought down several cell towers in the region. The network had been physically destroyed, severing connections and revenue. No algorithm, dashboard, or predictive model had accounted for an event like this.

I came across an article by David Brooks (Columnist at 纽约时报 ) on What Data Can't Do. In the article, the author covered the limitations of data analysis, and we need to be careful if we think of it as some panacea.

Data will always require human mind interpretation and context. Here are a few things you should be mindful of when doing data analysis:

Data Doesn’t Understand Context

Numbers tell you what happened but not why. My initial analysis showed revenue dropping, but without external insight, the cause remained invisible. Context matters, and sometimes, you need to step away from the numbers to find it.

Data Can’t Measure Emotion

Even when customers stopped using services, the data couldn’t capture their fear or frustration. It didn’t reflect the distress of people cut off from communication, unable to call their loved ones. Numbers alone don’t show human experience, and that’s where qualitative insight becomes essential.

Data Can’t Predict the Unpredictable

Historical trends are great; until reality throws something unprecedented your way. My forecasting models had worked flawlessly in the past, but they never anticipated an insurgent group taking down critical infrastructure. This was a stark reminder that while data can show trends, it can’t foresee black swan events.

Data Alone Won’t Make Ethical Decisions

In the aftermath, discussions arose about how to mitigate future risks. Should we reinforce infrastructure? Adjust our investments? The data couldn’t provide an answer; these were ethical and strategic decisions that required human judgment, not just numerical optimization.

Data Can Be Biased

Even in telecom, bias exists. The data I initially analyzed focused on urban areas, where revenue patterns were more predictable. Had I only looked at those trends, I might have missed the crisis unfolding in remote regions. Data collection is often skewed, reinforcing blind spots if not questioned critically.

Even with the best intentions, data can be misinterpreted. Correlation doesn't equal causation. A seemingly significant trend might be a statistical anomaly. Without a deep understanding of statistics and data analysis, it's easy to draw incorrect conclusions. Data literacy is crucial for navigating the world of numbers.

Data Can Be Misinterpreted

At first glance, the revenue drop looked like a simple market shift. Someone could have misattributed it to competitive losses or pricing issues without further investigation. Correlation does not equal causation; without proper analysis, even the best data can lead to false conclusions.

Data Doesn’t Tell the Whole Story

The data showed lost revenue but didn’t capture the bigger picture: the human impact, geopolitical reality, and operational challenges. It was a reminder that data is a tool, not a replacement for human insight, experience, and strategic thinking.

The Bottom Line

Data is a powerful tool, but it's essential to recognize its limitations. It's not a substitute for human judgment, critical thinking, empathy, or understanding of context. The most effective approach combines data insights with human intuition and a broader perspective on the world. Let's embrace the power of data, but let's not forget the things it can't do. The most valuable insights may lie beyond the numbers.

I write weekly on different topics related to Data and AI. Feel free to subscribe to FAQ on Data newsletter and/or follow Fawad Qureshi on LinkedIn or FawadQureshi on X.