Data over Opinions: Digital Experience Value #8

This current series addresses the need for clear set of guidelines for product teams in creating compelling and differentiated digital customer experience in B2B tech businesses. The seven previous editions drilled into seven core values that I believe businesses must embrace in the pursuit of distinctive competence in Digital Customer Experiences (DCX). This edition reviews the eight and last DCX ?value in the series - Data over Opinions.

Product teams must place deeper value on evidence-based decision-making over decisions driven solely by opinions and intuition. Establishing this value is foundational to nurturing a data-driven culture.

State of Product Decision Making

In an era where cloud solutions, IoT and digital twin models are the norm, it’s surprising to note that according to a recent TSIA poll, data is factored in a scant 11% of product roadmap decisions.

It’s not altogether surprising to see that Customers and Sales teams have high influence on the roadmap, however it’s distressing to note that the C-Suite, or HIPPO (Highest Paid Person’s Opinion), is driving many roadmap decisions. Is that the most optimal way to run a business? Do the executives in a company have a finger on the pulse of detailed customer behaviors more than the product and customer success teams whose very job it is to deeply understand the market trends and customer behaviors?

This survey data calls to mind the now famous quote from Netscape’s Jim Barksdale back in the 1990s. “If we have data, let’s look at the data. If all we have is opinions, let’s go with mine”

I love this quote because it succinctly encapsulates the tension between data-driven decision-making and subjective opinions and intuition. It highlights the absurdity of decision making based solely on (arguably) biased opinions and suppositions. ?When it comes to opinion, we all know the HIPPO will win 99% of the time.

Let’s review some examples of where beliefs based on data and opinion have played out in the market.

Products Built on Beliefs

There are numerous examples of products that were developed based on opinions or personal beliefs while downplaying data derived insights. While such products may succeed despite concrete data, many others face challenges due to their lack of alignment with market needs and user preferences. Here's few to ponder...

• Nokia and BlackBerry (2009): BlackBerry (formerly Research In Motion) and Nokia were both dominant players in the smartphone market. However, when touchscreen smartphones like the iPhone and the Android platform gained popularity, BlackBerry resisted the trend believing that physical keyboards were essential for business users and Nokia simply hesitated to act in the face of the data around them. ?This led to a delay in adapting to changing user preferences and contributed to the decline of both these companies.
• Google Glass (2014): Google Glass was a wearable technology with a head-mounted display. While the concept intrigued tech enthusiasts and developers, it faced challenges due to privacy concerns and the product's high price tag. The initial launch was more driven by Google's vision of the future than by concrete user research data, leading to limited adoption and eventual discontinuation. Google Glass was withdrawn in 2015.
• Quibi (2020), the shortform video streaming service designed for people to enjoy on their phones, has shuttered after less than a year of existence. While there were several factors that accelerated to Quibi's failure, key among them was not listening to the story the data was telling.

Conversely, companies like NI are leveraging test data to effectively offer visibility to system wide root causes, gain precision in failure identification and optimize predictive equipment maintenance. Reference Shelley Gretlein's Test Talk for an enlightening overview.

Where’s the Data?

How do you get it right when the HIPPO is flat wrong (e.g. Quibi's case above)? Who can argue with, or validate, the loudest opinions? Data can! And data sources abound. There’s no reason to ignore them, except of course, if tech business leaders, product management teams and founders believe they can dispense with available data insight. And a surprising number actually do.

Product teams must design in a wide range of data signals when planning their roadmaps to ensure their products are aligned with market trends, customer needs, and business goals. ?Here’s a snapshot of core data types for continuous consideration during the product lifecycle to optimize for success.

Collaboration between product, engineering, marketing, sales, support and customer success, is crucial to effectively gather, analyze, manage and consistently apply data from available sources.

Further, adapting the use of data signals to each lifecycle stage helps ensure that the products evolve in a way that meets user needs, aligns with business objectives, and maintains a competitive edge.

Ignoring or Minimizing the Data

Despite the availability of data, I have seen businesses choose not to leverage it in their decision-making process for various reasons. Why would that be? Some of the common factors contributing to this phenomenon include…

• A lack of a data-driven culture where leaders don’t invest adequately in data gathering and analysis.
• An over reliance on intuition and belief that personal experience and judgement is adequate.
• A lack of effective data strategies that lead to uncoordinated data acquisition, poor data quality, data overload and/or data analysis paralysis.
• The lack of data literacy leading to misunderstanding the data.
• A distrust of data quality leading to data disregard.
• Ignoring data that doesn’t align with the decision-maker’s theory or hypothesis.
• Ignoring or bypassing data analysis because of time constraints.
• Organizational silos making data sharing difficult, resulting in insights blind-spots.
• Fear of disruption that underscores resistance to decision-making process changes.

Do you recognize any of these in your organization? Creating a data-driven culture in your business starts with aligning on the value of Data over Opinions and promoting it loudly to all stakeholders. This provides the necessary permission to act.

Getting to a Data-Driven Culture

As the value of Data over Opinions takes hold in your organization, here’s a dozen probing data-driven questions to start with. These strategic and tactical question should be answered continuously to ensure that the value proposition aligns with customer needs, market trends, and the organization's goals:

1. Are we optimizing the use of all data sources in understanding the unique needs of customer segments?
2. Do our products have the right telemetry to understand the user behaviors across workflows and features?
3. Do we have systematic feedback loops for all product and all service engagements with our customers?
4. Are we effectively applying data from experiments to refine customer experiences and related value propositions?
5. How is quality test data providing visibility to system wide root causes?
6. How is usability test data systematically used to refine the user experience?
7. Are we inspecting the reviews, ratings and customer sentiment data of our competitors for deeper competitive landscape insights?
8. How are we deciding the optimal product and feature investments proposed products and features?
9. Are we effectively inspecting the investment and opportunity cost of technical debt and feature debt?
10. What is the cost of acquisition along the revenue lifecycle (RAC) for our products?
11. How does our (product or feature) value proposition impact customer retention rates?
12. What is the impact of regulatory or compliance data for our products and services?

What challenges have you experienced with bringing hard data to the decision making process? I'd love to hear your thoughts!