Data-informed but not Data-dictated | A case for bringing Imagination and Context in Product Development

Somewhere at the turn of the millennium, the decision-making processes in companies started witnessing a paradigm shift. Driven by a rise in internet penetration, digitization, superior data storage, access, and analytics infrastructure, the new age came to be heralded as the age of data-driven decision-making. And over the years, maxims around Data usage started popping up, examples being:

a) Data is the new Oil.

b) In God we trust, others must bring data.

c) Without big data, you are blind and deaf, and in the middle of a freeway.

Now as someone who has worked with numbers for a living, I am a staunch believer in the efficiencies realized through data-informed decision-making. That belief continues to be strengthened as the world gets more and more digitized. No surprises there.

That said, over the years, I have also come to observe that it takes a certain judgment call to contextualize and derive insights from the data gathered. Solely relying on data sans the context doesn't always lead to the glory land of meaningful insights. And the absence of data doesn't necessarily invalidate a concept.

So in the subsequent paragraphs, I would argue why it's prudent to consider and place due value on the softer stuff of imagination, leap of thoughts, and contextualization into what the future may look like. And why supplementing the softer stuff of imagination, first principles with the hard stuff of data is probably the winning recipe.

---------------------------------------------------------------------------------------------------------------

Data doesn't exist for category creators

So like every other millennial working in internet startups, drinking black coffee, and running a SubStack newsletter - I have also done my bidding of Sapiens, Zero to One, biographies on Musk, Jack Ma, Phil Knight, etc.

The point from the above humblebrag is that these books present a key insight as a common theme. That insight is the founders taking their big bets on a mix of foresight and first principles thinking. The bets evolved over the years and became their current versions using data and multiple iterations. In the germination of the broader idea, however, the role of market research data was quite limited.

Let's take a moment. I am sure if you have read thus far, you probably know about Ritesh Agarwal. Now Ritesh hardly did any formal 'landscaping' study on OYO before starting. Basis his personal experiences of staying in different places and some ground study, Oravel started as a B&B concept. The concept later evolved into OYO which started with offering budget accommodation and now they do everything and anything in hospitality. A decade back, a concept like OYO was difficult to even fathom, let alone have some formal research done behind it.

Similarly, there was not much data on ride-hailing as a sector before Ola, Uber, Grab, Lyft came into play. So if you wanted to venture into on-demand ride-hailing and were looking for 'validation through data', there was hardly anything substantive. The market size would have seemed exponentially lower than what it has turned out.

Tesla is another example. Elon Musk started Tesla 10+ years back when all the existing analysis suggested he torch up the money rather than put in an electric car company. As it turns out, a decade later, it is the most valuable automotive company in the world. More importantly, Tesla has been the node of most of the major automotive companies taking notice of Electric cars as a commercially viable product and driving the industry forward.

The key point is that formal market validations for category creators aren't necessarily available at the time of their germination. And the absence of data by itself doesn't necessarily invalidate the idea.

Data signals sometimes turn out to be incorrect

Yes, that also happens once in a while. Even if we have multiple Excel sheets suggesting a certain outcome, that may not turn out to be always true. Sometimes it's just bad analysis, sometimes it's noise from early adopters construed as the general insight, and sometimes an amalgamation of incorrect biases and assumptions. I also find this to be an uncomfortable reality to grasp - genuinely wishing life was simpler and mangoes were available all year round. But that's life. Here are a few examples to that end.

• In 2010-11, Facebook created the News Feed product where you could show off the avocado toasts (before Instagram gobbled it all up, pun very much intended). The product received an infamous backlash from the users back then. It seemed almost everyone hated this update . Looking at the numbers, reversing back the product would have been the most apparent decision - because hey hey, the users don't want this, right? And as it turns out, News Feed became the de-facto method of sharing content on Facebook after a few months and a decade to come.
• Taking Amazon as a case study, there is a famous anecdote where the internal teams didn't see any financial viability in launching Amazon Prime. Bezos, despite the pushback and analysis, trusted his imagination and focus on customer obsessions than the financial projections. As it turns out, Prime is now the biggest loyalty program in the world ten years later. You can read that story here .
• And on that note, let's talk about video conferencing. None of the formal studies would have identified a huge market for another video-conferencing tool when the behemoths like Google / Microsoft / Cisco / Apple had their video conferencing solutions that worked well enough. However, a person named Eric Yuan chose a contrarian stance and created a $100 Billion company called Zoom where we peek into our colleagues' living rooms while wearing sweatpants.

So here we go. Sometimes, the conclusions from the analysis don't always work - a reason for which makes my third argument below.

Data is a scorecard of historical competencies and tastes

• When electricity was first generated through Solar Energy in the late 1880s, solar energy efficiency was < 1% (less than 1% of Solar energy received on the panels was converted into electricity). It made electricity generation through Solar energy utterly unviable as Solar energy couldn't simply compete with fossil fuels. A century and tons of research and implementation later, Solar energy is now at 20% efficiency and powers 2% of the world's electricity. The primary energy source of satellites is solar energy. Sure there is a long way to go but the point is if everyone in the 1880s would have looked at the Data available and internalized it to be the holy truth applicable for the future as well - we would never have gotten Solar energy to come this far.
• Another example is the evolution of digital products built on the philosophy of mobile-first experience. It may seem ancient but just 10 years ago, most of the sleek apps we use daily didn't even exist. The smartphone penetration in India was ~ 3% in 2011 which currently is above 50% - not to mention the leaps and bounds by which the hardware and software have improved. Julie Zhuo, founder of Sundial and an ex-VP Product at Facebook, summed it really well here on this phenomenon.

"The power of big bets: no metric can tell you what the bold strokes needed to win the future are. Imagine 2008, when smartphones were just starting to emerge. If you looked at the metrics for your website, you would have seen a tiny sliver of traffic coming from smartphones. You may have concluded, very practically, that you shouldn’t really invest too much into building for mobile since it’s such a small part of your audience. Today, we realize the vision and foresight of those who did bet big on mobile and reaped huge rewards. No examination of current behavior can accurately tell which way you need to leap. "

• And just so we don't talk only about internet-based companies, let's discuss the case of Red Bull. Founders of this iconic brand faced repeated rejections from potential investors who thought that the absence of an Energy Drink category invalidated a product like Red Bull for European markets. And we all know how well that stance turned out to be. The classic example of the absence of a product or category and hence - related data - doesn't necessarily invalidate its need. Similarly, in the early 20th century when Wright Brothers were tinkering to build a flying machine, they didn't have access to a market study by a consultancy firm written in an expensive-looking font. It was the imagination to build a flying machine grounded in solid fundamentals of physics.

So just to reiterate the point - the phenomenon that looks fringe today may become mainstream tomorrow. We don't know for certain which ones would and that's where the hard stuff is. VCs have to make these judgment calls regularly and despite having access to one of the most privileged networks, they make mistakes too. Bessemer Ventures famously has a repository of companies they passed on from investing including Google, Apple, Facebook, PayPal! So a lot of smart folks are susceptible to this.

Public figures exhorting the value of imagination for big bets

In the above paragraphs, we have covered some of the reasons on where to place due importance on imagination, first principles, and intuition. To this end, I also wanted to check if some of the public figures in business have talked about their decision-making. The below section covers the endorsement of imagination and flair for step function innovation.

David Rubenstein, the co-founder, and chairman of The Carlyle Group hosts a business version of Conan with well-known global leaders. Now in one of the interviews, Rubinstein was hosting Jeff Bezos. So as any other person aiming to become rich and famous and subsequently quoting Jim Carrey when I am 60, I was holding onto each word coming out of Jeff Bezos. There are multiple aphorisms shared by Bezos in the interview. However, let me share the part which sets the premise of this post. On how Bezos makes decisions, the quote below :

"All of my best decisions in business and in life have been made with heart, intuition, guts, and not analysis. When you can decide with analysis, you should do so, but it turns out [that] in life, your most important decisions are always made with instinct, intuition, taste, heart and that's what we will do with this Day One Foundation to it."

Now wait, wait, wait. Did we just read that the CEO of Amazon makes his big bold bets through the 'flimsy' concepts of intuition, guts, and taste? What are we in, 1985?

Yes, we did. Bezos does place a high value on the decisions on these 'flimsy' concepts. And mind you, he isn't yet another hippie high on fashionable nonsense by Sadhguru. He is the founder and CEO of Amazon which is as rich with data as it gets. So, it's serious.

Extending on Bezos's view of solving customer problems, he views only listening to customers doesn't help much, the invention of a solution is what brings non-linear returns.

From his shareholder letters of 2010 and 2018 respectively:

"Start with customers, and work backward. Listen to customers, but don’t just listen to customers – also invent on their behalf."

"No customer was asking for Echo. This was definitely us wandering. Market research doesn’t help. If you had gone to a customer in 2013 and said “Would you like a black, always-on cylinder in your kitchen about the size of a Pringles can that you can talk to and ask questions, that also turns on your lights and plays music?” I guarantee you they’d have looked at you strangely and said “No, thank you.”

Let's take another example. Steve Jobs in his now massively popular speech at Stanford, had the following excerpt below:

"Your time is limited, so don’t waste it living someone else’s life. Don’t be trapped by dogma which is living with the results of other people’s thinking. Don’t let the noise of others’ opinions drown out your own inner voice. And the most important is to have the courage to follow your heart and intuition. They somehow already know what you truly want to become. Everything else is secondary".

See a similar adage on following the heart and intuition? This speech got so popular that every 'thought leader' on LinkedIn has used it once for easy virality. That R-square would make covid jealous.

---------------------------------------------------------------------------------------------------------------

With all the case studies and arguments above, I wanted to put the torch on how and when solely relying on Data without context can have its limitations. As someone who has worshipped at the altar of data for many years now, I now look at Analytics mostly from the point of optimization. Achieving local-maxima at regular intervals rather than validating for step-function jumps. Let's cover this below in the following example.

Data is a powerful tool for Optimizations and fine-tuning

Say you sell pens. There is a particular pen that you sell at Rs 100 and on average, you sell 10 units. You decide to experiment with the pricing and sell at Rs 150. You see that you are still able to sell 8 units a day on average. With Rs 200 however, the sell-through is only 4 units. The revenue generated in the 2nd case is the maximum and if that was your business goal, the price point of Rs 150 is the best for you. Voila!

This is an over-simplified example of how some A/B testing and data can help you price your product to optimize revenue.

The use-cases of data are multi-fold for optimization and fine-tuning. It can help in answering questions like:

a) Why the customer returns of a certain E-commerce category are higher and what can we do to reduce the return percent?

b) Which logistics provider would be cheaper to ship from place A to place B?

c) How do we optimize the online checkout process?

d) Do we have traffic, conversion, or retention problem based on industry benchmarks?

And questions like these should continue to be driven by data-informed decision-making. Data can also help in informing the decision around which markets and products to launch but it can't dictate the decision-making process.

So what does it mean for Product Development?

That the rhetoric of data-driven decision-making or data-driven product development should be contextualized.

That we don't make the product development process so regimented that there's no space for imagination or ingenuity or first principle-thinking to flourish.

That next time we're trying to build a new product that has a certain amount of novelty, let the first principles come in, mingle with some of that imagination, and contextualize the data available. And see this "imagination" thing isn't a magical entity operating out of a vacuum. It's an abstraction of life experiences, leap of thoughts, heuristics developed from analysis and learnings.

Hence - just to re-emphasize, the post aims to bring the nuance in treating data for milking the insights. That Data, despite all the benefits it holds, has its limitations. Especially, when it comes to envisioning step-function products. It isn't about dissuading people from doing due diligence and add 'Visionary' to our LinkedIn profiles. The idea, rather, is to share a framework where both Data, Intuition, First-principles, Experimentation can co-exist to complement each other.

Just like making decisions only based on intuition and faith is dangerous, relying solely on historical data sans context and considering the absence of data as a tacit invalidation of concept is also incorrect. The so-called 'success' of a product is a concoction of Imagination, Faith, Experimentation, First-principles, Execution, Analytics, and Luck.

Thank you for reading this far. All feedback is welcome.

References for further reading

a) https://hbr.org/2021/03/data-is-great-but-its-not-a-replacement-for-talking-to-customers

b) https://www.forbes.com/sites/karlsun/2018/06/26/the-decision-making-dilemma-when-to-trust-your-gut-vs-the-data/#79b985611041

a) https://productlessons.substack.com/p/where-data-ends-and-intuition-begins

d) https://www.entrepreneur.com/article/222501

e) https://thesnippet.substack.com/p/dont-listen-to-your-customers