LOVE THE PROBLEM, NOT THE SOLUTION: WHY MARKET RESEARCH IS IMPORTANT FOR PRODUCT DEVELOPMENT

New Coke: The year was 1985. Pepsi had launched a marketing campaign about how consumers preferred Pepsi in taste over Coca Cola. Pepsi and Coke were in fierce competition. They were going at it eyeball to eyeball.Coke was still the market leader but its market share was in decline.

Coca Cola analyzed the situation and conducted its own surveys. They revealed the same story: Consumers liked Pepsi better in taste. Coca Cola panicked at this point and took two bold steps: 

1. After 99 years, they decided to change the recipe, to match the taste of Pepsi which was sweeter as compared to Coca Cola. 
2. They recalled complete stock from retailers across United States and filled freezers with “New Coke”. Yes, the name was different and the taste was much like Pepsi. 

However, It turned out that surveys were conducted through Sip testing. People were asked to take a sip and share their feedback. When it came to consuming the whole can, Coke was still preferred choice. Also, the surveys were not conducted in consumer settings where they would drink refreshing Soda at home or with friends. People behave differently in simulated environments or artificial arrangements. 

Coca Cola learned a lesson here. After experiencing horrible sales for two and a half months, they recalled the stock again, and brought consumers favorite soda: the original Coca Cola. 

Here is what Coca Cola president admitted later: 

"The simple fact is that all of the time and money and skill poured into consumer research on a new Coca-Cola could not measure or reveal the depth and abiding emotional attachment to original Coca-Cola felt by so many people" 

Coca Cola said recently:

“When we look back, this was the pivotal moment when we learned that fiercely loyal consumers -- not the Company -- own Coca-Cola and all of our brands” 

Touch of Yogurt shampoo: A few years prior to this, P&G decided to respond to “Back to Nature” movement which insisted on using natural ingredients for shampoo and other products. P&G believed it should add dairy ingredients to its shampoo products. It launched “Touch of Yogurt Shampoo”. It failed. People could not associate dairy product with shampoo even though they were looking for natural ingredients. People even perceived it to be a Yogurt product. There were cases of consumers eating the product assuming it to be Yogurt. P&G did not specifically ask customers if they associated dairy products with shampoo. 

If you look at a few other spectacular product failures, You will notice that these billion dollar companies occasionally launch their products without enough market research. Microsoft launched Zune to compete with Ipod, only to learn that Ipod already had 86% market share by then and People saw no reason to use another Metoo product. 

Amazon launched fire phone only to see it rolled back. Ford launched their infamous Edsel in 1950s which cost them around $3 billion (adjusting time value of money). HP launched Touchpad tablet to compete IPAD, it managed to sell only a few thousand copies. Apple launched Newton to compete other PDAs at the time. All of these were decent attempts with reasonable product outcome, yet all of them failed massively. The companies behind these products had billions of dollars in revenue, yet they could not figure out what customers wanted.

We will analyse why products fail and what exactly is the problem here.

So how many products fail? 

• Neilson says: Approximately one half of innovations tested by Nielsen BASES don’t effectively articulate to consumers how they deliver on a broad consumer need. 
• According to Clayton Christensen of Harvard business school, There are over 30,000 new products introduced every year, and 95 percent fail.
• McKinsey global institute puts it this way: Out of every 7 product launches, only one succeeds. Less than 3% of newly launched products do not exceed $50 million in sales.

And why do so many of them fail: 

Mckinzey puts the reason of failures here *4: More than 80 percent of the top performers said they periodically tested and validated customer preferences during the development process, compared with just 43 percent of bottom performers. 

They were also twice as likely as the laggards to research what, exactly, customers wanted. That made them better able to identify and fix design concerns early on, minimizing project delays.

This info-graphic explains why you should read the rest of this article carefully:

A few reasons of failure which can be traced to lack of market research are: 

1. No Market Need
2. Poor Product
3. Ignore customers
4. Product mis-timed

Going through the list, why is it that someone would develop something with no market need at all? Entrepreneurs put their skin into the game. They leave out on lucrative opportunities only to be called outliers. But then Why would someone develop a mistimed or poor product? The answer lies in:

“Loving the solution, rather than the problem” 

We have seen dozens of startups fail because they just loved the solution they were offering. Stephen Covey says that everything is created two times: First in the mind and then in reality. If you are to build a house, You will draw it on a paper or software with architects. Any changes to design will be costly once you start implementing it. It may ruin plans altogether.

The same is true for businesses. If enough market and customer research is not done before launching the product, You might end up with 95% who are developing failed products. That's reality. Better face it now and set the first design right. 

So how market research helps? It helps you in two things: 

1. Validating the existing plan 
2. Coming up with new business and marketing plan 

Precisely, it helps you answering questions such as:

• How many potential customers are out there? 
• How much price are they willing to pay? 
• How many products I can sell?
• How does market look like? Is it already Saturated? 
• Where I can position my product?
• Can I do cross selling or up selling with my product?

Launching a new product? How market research helps! 

Suppose you are launching a new grocery brand in US and You want to sell your product at $3 each piece. You have finished the product and now ready to take it to stores. You go to retail stores and ask them to place it along with other products at $3. Is that alright? 

Now if you do a little research and come across this trend of US consumers: 

So most of consumers want to try a new product at discounted price or with a coupon. There are some curious customers who might buy it anyway. Given this secondary market research (done by someone else, but it still greatly matters), You can launch the product at discounted price or by giving coupons or sampling product at community events and other places. Do you think your chances of success will be boosted?

GETTING STARTED ON MARKET RESEARCH 

I hope you have understood the importance of market research by now. There are 2 ways you can go about it: 

1. Primary research 
2. Secondary research 

Primary research is what you do yourself for your product. That includes doing surveys, focus groups and interviews with potential users. 

Secondary research means relying on other sources to seek data and analysis of your potential users. That includes data collection from government agencies, research agencies, online sources and trade associations. 

A great market research involves both primary and secondary sources. Even if you have best quality secondary research available and you believe a great product can be developed based on that, you will still need to talk to your potential customers. They are buying from competition already or they are using alternate methods to do the same job. You need to know:

• Gender, Age, Marital status, Ethnicity and other details of your potential users 
• Where do they live? 
• What is their economic Status? 
• Do they already buy? How do they buy? 
• What are Issues about your competition?
• What do they like about your competition?
• What are they paying now? 

Don’t assume people landed from Mars today, they are using a competitor's product or an alternate already

And when you are talking to them, do what Lawrence Grobel offered: 

“Converse like a talkshow host, think like a writer, understand subtext like a psychiatrist and have a ear like a musician” 

If you are launching something for mass market (or consumers), You wont get a chance to do hundreds of interviews. What you can do instead:

1. Make groups of people who will be using your product. Let say, students or professionals, Age, Gender, SEC segments etc 
2. Interview each group and get insights accordingly. That will help a lot defining survey for everyone. 
3. Remember the minimum sample size for probability distribution? It was n=30. Remember that in mind. I’d suggest, have 100 responses at the minimum. 

Having said that, lets understand the concept of TAM,SAM and SOM here: 

TAM: In an ideal world, everyone wants to acquire every single potential user out there. That's dream of every entrepreneurs or every single product team. That’s what we call TAM (Total available market). 

SAM: But reality is slightly different. In reality, its not possible to reach out to everyone in TAM. You have geographic limitations. Your product may not be for everyone. So the market which you can really target is SAM (Served Addressable market or Feasible market), which is subset of TAM.

SOM: And we all know you cannot get everyone to buy your product. So if you can reach 100 people, may be few of them will like and purchase your product. That's your share of market (SOM) which is subset of SAM. 

Here is an example from AirBnB. This is part of their presentation they made in YC in early days.

It says, 1.9 billion trips are booked every year (TAM)

We can target 25% of those who book it and they are budget travelers who book online (SAM)

Out of 25%, we can make 2 customers out of every 100 people we target (SOM)

In other words, Our market share (SOM) will be one out of every 50 we target (2% of SAM) and 1 out of 200 global budget travelers (0.5% of TAM). 

Lets take another example. Suppose you are developing a robot to do food automation, say Pizzas. So you want to sell it to every pizza maker on earth, right? Wait, why not every restaurant? Once they have the automated machine, they won’t have to hire a specialized chef to bake pizzas (Making food is chemistry).

So ideally you want to sell it to every restaurant across globe. That’s your dream. That's TAM.

Can you sell it to every single restaurant on the globe? May be you would want to become top player in your country or region, and want to target them first for next 3-5 years. Many times, You are constrained by geographic limitations. May be it is product constraints or customer constraints. Incase of pizza making robot, you want to sell it to maximum people. However, the budget constraints only allow you to reach out restaurants in your area or country. So the market which we can target in reality in SAM. 

Out of every 100 restaurants, You might sell pizza maker to 1 or 2 of them. That's your share of market. 

So how do we calculate TAM, SAM and SOM: 

Consider the following factors when you calculate TAM: 

• Current market Size (How big the market currently is)
• Structure (How potential users can be segmented into categories)
• Growth prospects (Is the market expanding due to certain events or is it shrinking)
• Trends (What are user trends, how external factors are influencing the market)
• Sales Potential (Do potential customers have enough resources to buy your product) 

You can get the market size from secondary data such as govt agencies, market research companies or statistic bureaus. 

The market structure can be obtained both from secondary and primary market research. The current available surveys might point out to how potential users can be classified based on different criteria. Alternatively, You can talk to end customers through in person surveys, online surveys and focus group interviews, and You can see how they can be classified into different groups. You can find out sales potential from this activity. 

Trends and growth prospects can be gauged by monitoring digital trends (Google trends, Twitter, Youtube etc). Keep an eye on different public opinion survey agencies. 

SAM can be taken out of TAM considering three factors: 

1. Geographic location (You cannot reach out to everyone in TAM due to geographic limitations)
2. Product or service attributes (Your product won’t be for everyone in your TAM). Incase of AirBnB, Brian Chesky and Co might have thought if you were not budget traveler, You might not want the product (However, the trend has gradually changed across years) 
3. Customer attributes (Will an iPhone loyal customer consider other brands?)

SOM is defined as share of market, and it depends on your business strategy. A few important factors are: 

1. Pricing strategy: How you price your product
2. Distribution strategy: How your plan to take it to your customer
3. Marketing strategy: How you plan to make your customers aware of your product
4. User adoption and profitability strategy: Do you plan aggressive user adoption? Do you plan profitability and compromising on fast growth? Do you plan a mix of both? 
5. Manufacturing strategy: Can you smartly manufacture something at low cost and/or better features that your competitors cannot
6. Innovation / R&D strategy: Are you setting high barrier to entry by introducing an innovative product which requires lot of R&D

To understand SOM, lets go back to Pizza maker example, you have figured out that you will reach out to pizza makers in your country (SAM). You start contacting pizza shop owners. Of every 10 shops you contact, you successfully establish contact with 2. So that’s 20% of SAM. Every 1 in 10 persons who listen to you ask for further information or demo. So you establish that 2% pizza shops in your country ask for demo. Once this group of 2% pizza shop owners try, 25% of them become your customers. So you have 0.5% success ratio which is SOM.

Next time you plan to develop a product, I would suggest do the following: 

1. Don’t act on impulse. Think about it for really long time. If you are working in teams, don’t group-think. Try to find empirical evidence for assumptions you are basing your product upon. 
2. Talk to customers. Find out who they are?
3. Love the problem, not the solution. 
4. Make a problem statement to solve by talking to as many potential customers as possible. Like many other things, it’s a funnel. You will talk to many potential customers if you talk to lots and lots of people! 

Good Luck. And remember, Love the problem, not the solution!