Is this the best AI-powered market research approach? – with Carmel Dibner
Chad McAllister, PhD
Product management professor, practitioner, and top-1% podcast host. I help product managers and leaders prepare for the next step in their careers.
To listen to the interview, search for Product Mastery Now on your favorite podcast player.
How AI captures customer needs that humans miss
Watch on YouTube
TLDR
In my recent conversation with Carmel Dibner from Applied Marketing Science, we explored how artificial intelligence is transforming Voice of the Customer (VOC) research for product teams. The collaboration between AMS and MIT researchers has yielded impressive results, with AI tools not only matching human analysts in identifying customer needs but often exceeding them—especially for emotional needs that humans might overlook. Rather than replacing human researchers, AI serves as a copilot, helping product teams uncover twice as many unique needs while reducing analysis time and eliminating bias. This hybrid approach offers tremendous potential for innovation, particularly in the early stages of product development.
Key Topics
Introduction
Voice of the Customer research has been a cornerstone of product management for decades. But it is changing, with AI tools that are transforming how we uncover and analyze customer needs. While some fear AI might miss the human element of customer research, recent advancements show it can actually help us capture more nuanced emotional needs while eliminating human bias.
Joining us is returning guest, Carmel Dibner, who is a principal and co-owner at Applied Marketing Science (AMS), where she has helped companies uncover critical customer insights to improve products, services, and customer experiences.?Before moving to consulting she was in brand management at Unilever. More recently, she has collaborated with AI researchers at MIT to improve VOC outcomes. I regard Applied Marketing Science, Carmel’s company, as the thought leaders in VOC research, and it was the first organization to formalize the VOC interview process.
In this discussion, we’ll explore how LLMs are revolutionizing Voice of the Customer analysis. Carmel will share results of experiments where AI not only matched human analysts in extracting customer insights but excelled at finding hidden needs – unmet needs that could unlock your next innovation opportunity and create competitive advantage.
Whether you’re skeptical about AI in customer research or eager to embrace it, this discussion will challenge your assumptions about the future of Voice of the Customer analysis.
The AI Revolution in Voice of the Customer Research
Early AI Experiments (2017-2018)
AMS began experimenting with artificial intelligence for customer research around 2017-2018. Their initial focus was on developing algorithms that could effectively analyze textual data and extract meaningful customer insights.
However, these early efforts faced significant limitations. The AI could identify potentially useful information, but human analysts still needed to invest considerable time sifting through and making sense of what the AI had found. The process wasn’t yet efficient enough to deliver the time-saving benefits they hoped for.
Recent Breakthroughs (2023)
In 2023, AMS and MIT researchers tackled a more ambitious question: Could AI now effectively craft unmet customer needs statements that would be just as good as those created by experienced human analysts?
To answer this question, they employed a technology called supervised fine-tuning. This approach involved “teaching” large language models how to craft clear, actionable customer needs statements based on transcript data, social media comments, and other text sources.
The supervised fine-tuning approach represented a significant advancement. Rather than simply flagging potentially relevant text, these newer AI models could produce fully formed needs statements that captured what customers truly wanted and why.
This breakthrough laid the groundwork for the impressive results they observed in their comparative experiments between AI and human analysts.
Validating AI Effectiveness in VOC Research
The claim that AI could match or even exceed human performance in VOC research required solid evidence. During our conversation, Carmel described several experiments they conducted to validate the effectiveness of their AI approach.
Experimental Approach
One of their key experiments involved a blind testing methodology. They took authentic customer needs statements from previous VOC studies conducted by human analysts and mixed them with needs statements that the AI had generated. Then, they asked experienced human analysts to evaluate all the statements without knowing which were AI-generated and which were human-generated.
The analysts evaluated each statement based on several criteria:
Surprising Results
In these blind tests, the AI-generated needs statements performed just as well as—and in some cases better than—those crafted by human analysts.
Key Benefits of AI-Powered Analysis
Through these experiments, Carmel and her team identified several significant advantages of using AI for VOC research:
This last point about reduced bias is particularly important. As product managers, we sometimes unconsciously look for evidence that confirms our existing assumptions about customer needs or product direction. An AI system, properly implemented, doesn’t have these same motivations—it simply reports what it finds in the data.
These benefits combine to create a more robust database of customer needs, which serves as the foundation for effective product innovation. The faster a product team can build this comprehensive understanding of customer needs, the more quickly they can move into solution development with confidence.
AI’s Ability to Capture Emotional Needs
One of the most surprising findings from AMS’s research was how effectively AI could identify emotion-infused customer needs. This discovery challenged a common assumption that machines would struggle with the emotional aspects of customer research due to their lack of human empathy.
Why AI Excels at Finding Emotional Needs
Humans conducting customer research are often unconsciously biased toward functional needs. As product professionals, we’re trained to identify problems and create solutions. We get rewarded professionally for finding practical issues that can be addressed with concrete features or improvements.
This solution-oriented mindset can cause us to quickly move from emotional needs to functional needs to potential solutions. It’s simply how our professional brains are wired.
AI, however, doesn’t have this bias. It gives equal weight to functional and emotional needs in customer data because it isn’t influenced by the pressure to jump to solutions. This creates a unique advantage in identifying the full spectrum of customer needs.
Case Study: Wood Stains Category
In analyzing customer feedback about staining furniture and wood products, the AI identified an emotional need that human analysts completely overlooked: Customers wanted “a manufacturer that values my feedback, will respond to my emails, and will address my concerns.”
Human analysts had dismissed this as a generic desire that everyone would have, not recognizing it as a core need specific to this category. However, this need is particularly important for wood staining projects because customers often encounter problems and need responsive manufacturer support.
The AI, without bias toward “important” functional needs, recognized this emotional need as significant based purely on the data.
The Competitive Value of Emotional Needs
There are three dimensions to any customer job:
While functional needs are often the easiest to identify and address, emotional needs frequently drive purchasing decisions and brand loyalty. A product that connects strongly with customers’ emotional needs will typically outperform one that only addresses functional requirements.
Even in highly functional categories like home heating and cooling systems, emotional needs like “feeling like a responsible homeowner” or “not feeling like I’m throwing money down the drain” are important to customer satisfaction.
By leveraging AI to help identify these often-overlooked emotional needs, product teams can develop more holistic solutions that connect with customers on multiple levels, creating stronger competitive advantages.
The Human-AI Partnership in VOC Research
While the results from AI-powered analysis are impressive, Carmel emphasized that the most effective approach is using AI as a copilot rather than a complete replacement for human researchers. This partnership model leverages the strengths of both human expertise and AI capabilities to produce superior results.
AI as a Copilot
During our conversation, Carmel described how she views the relationship between human analysts and AI tools. The AI will find many things that humans overlook, but humans will also identify aspects that AI might miss. It’s similar to having two different analysts review the same data—each will notice different things. This copilot perspective emphasizes collaboration rather than replacement.
Quantifiable Advantages of the Hybrid Approach
The partnership between human researchers and AI creates measurable benefits for product teams. In their experiments, AMS examined the overlap between needs identified by humans and those identified by AI, creating a Venn diagram of findings.
The results were eye-opening:
These niche needs—ones that might be mentioned infrequently in customer feedback—are opportunities for innovation. As Carmel pointed out, the frequency with which a need is mentioned doesn’t necessarily correlate with its importance. Sometimes, these rarely mentioned needs represent the greatest opportunities for competitive differentiation.
Case Study: Finding New Insights in Mature Industries
To illustrate how the human-AI partnership can unlock unexpected value, Carmel shared an example from the snowplow industry. She worked with a client who had been in the snowplow and snow equipment business for 20 years—a veteran who deeply understood the industry.
Initially, she wondered what new insights they could possibly discover for someone with such extensive experience. However, through their research process, they uncovered insights about visibility issues—specifically, that snowplows often have the worst visibility precisely when they need it most.
These insights weren’t obvious even to industry experts, but they represented significant innovation opportunities. The client had never thought to ask certain questions, but the AI-augmented research process helped uncover these hidden needs.
Breaking Through Subject Matter Expert Limitations
AI-powered research can help overcome the limitations that come with deep domain expertise. When we’re very familiar with an industry, product, or customer base, we often make assumptions that limit our perspective. We might believe we’re in a commodity space or that we already understand all customer needs.
In these situations, an outside perspective is valuable. Traditionally, this might come from new team members who aren’t constrained by industry conventions. Now, AI tools can provide a similar fresh perspective, almost like bringing in a consultant who specializes in finding customer needs across many different categories and can apply that expertise to your specific domain.
Limitations of AI in VOC Research
Despite the impressive capabilities of AI in VOC research, it’s important for product teams to understand its current limitations. During our conversation, Carmel highlighted several areas where human judgment and expertise remain essential.
Equal Weighting of Data Sources
One significant limitation is that AI algorithms like large language models tend to treat all data sources with equal weight. As Carmel explained, these tools currently can’t make sophisticated judgments about the relative validity or reliability of different data sources they analyze.
This means that human expertise is still necessary for:
The AI is only as good as the data provided to it, so human selection and curation of input data is critical.
The Continued Value of Direct Customer Conversations
There’s still no substitute for real, live conversations with customers. While AI can extract tremendous value from interview transcripts and written feedback, direct customer interactions provide benefits that go beyond data collection:
These human-to-human interactions develop deeper understanding within product teams that purely AI-mediated research might miss. The ideal approach combines direct customer conversations with AI-powered analysis of the resulting data.
Prioritization Challenges
Carmel also noted that AI currently can’t prioritize needs for innovation effectively. While AI excels at identifying the full spectrum of customer needs, it doesn’t yet have the capability to determine which of those needs represent the most valuable opportunities.
The AI can identify all the needs that should go into a prioritization survey (what AMS calls “secondary needs”), but product teams still need to:
This final step of prioritizing where to focus innovation efforts remains a human-driven process that requires business judgment, market understanding, and strategic thinking.
Balancing AI Capabilities with Human Expertise
Understanding these limitations helps product teams use AI tools more effectively. Rather than seeing AI as a complete replacement for traditional customer research methods, the most successful approach treats AI as one powerful tool in the product manager’s toolkit.
By being realistic about what AI can and can’t currently do, product teams can design research processes that leverage the strengths of both AI analysis and human expertise, creating more comprehensive customer insights than either could achieve alone.
Integrating AI into the Product Innovation Process
With an understanding of both the capabilities and limitations of AI in VOC research, the next question becomes how product teams can effectively incorporate these tools into their innovation processes. Carmel shared several practical insights on this topic during our conversation.
AI Throughout the Innovation Funnel
Carmel described AI as providing product teams with “different lenses” to view customer needs. Unlike traditional research projects that often require extensive planning and formal structure, AI-powered approaches offer more agility and flexibility.
Product teams can leverage AI at multiple stages of the innovation funnel:
Most Useful Application: Beginning of Innovation
While AI can provide value throughout the innovation process, Carmel emphasized that its most useful application is at the very beginning—the discovery phase where teams are trying to understand the landscape of customer needs before diving into solutions.
This aligns with best practices in product management, where thorough understanding of customer problems should precede solution development. AI can help product teams build this foundation more quickly and comprehensively than traditional methods alone.
Filling Gaps in the Process
AI can fill gaps when the ideal innovation process hasn’t been followed. In practice, product development doesn’t always follow a perfect linear path:
In these situations, AI can quickly analyze customer data to ensure teams haven’t missed anything foundational before proceeding to later stages. The speed and efficiency of AI analysis makes it a highly agile tool for course correction.
Broadening Innovation Horizons
Another benefit of integrating AI into the product innovation process is how it can help teams break out of established patterns of thinking. By identifying needs that might be overlooked in conventional analysis, AI can point product teams toward unexpected innovation opportunities.
This is particularly valuable in mature product categories or for teams working with products they’ve managed for a long time. The AI can help challenge assumptions and reveal new possibilities that might not have been considered.
Practical Implementation of AI-Powered VOC
Understanding the potential of AI in VOC research is one thing, but knowing how to practically implement these tools is another challenge entirely. During our conversation, Carmel provided insights into how product teams can begin leveraging these capabilities today.
Current State: Client Service Model
Currently, AMS is offering AI-powered VOC as a service to their clients. This approach allows product teams to benefit from advanced AI analysis without needing to develop the expertise or tools internally. Carmel explained that their goal is to help clients get insights faster and more efficiently than ever before.
This service model makes sense given the specialized nature of the AI tools being used. The large language models employed by AMS have been fine-tuned with supervised learning based on 1,500 carefully selected customer needs from multiple case studies. This specialized training creates AI systems that are specifically optimized for VOC research rather than general-purpose AI.
Benefits for Product Teams
Implementing AI-powered VOC research, whether through a service provider like AMS or eventually through internal capabilities, offers several practical benefits for product teams:
Tailoring Implementation to Product Complexity
Carmel noted that the level of detail needed from AI-powered analysis varies based on the complexity of the product and its development stage. Not all products require the same depth of customer needs exploration:
The flexibility of AI-powered approaches allows teams to adjust the scope and depth of analysis to match their specific situation, making this a highly adaptable tool for diverse product development contexts.
Conclusion
The integration of AI into Voice of the Customer research represents a significant advancement for product teams seeking to better understand and address customer needs. These tools aren’t replacing human researchers but rather enhancing their capabilities, helping teams discover more customer needs—particularly emotional ones—faster and more comprehensively than traditional methods alone. The ability to process diverse data sources without fatigue or bias opens new possibilities for innovation, especially in mature product categories where fresh insights can be challenging to find.
By leveraging AI to help us better understand customer needs—both functional and emotional—product teams can create solutions that connect more deeply with customers, driving both satisfaction and competitive advantage.
Useful Links
Innovation Quote
“Simplicity is the ultimate sophistication.” – attributed to Leonardo da Vinci
Application Questions
Bio
Carmel Dibner is a principal in the Insights for Innovation practice at Applied Marketing Science (AMS) where she is responsible for client relationships, client service delivery, and business development.
She has worked closely with researchers at the MIT Sloan School of Management to experiment with new AI techniques and has successfully applied machine learning to answer her clients’ most difficult research questions. These techniques became the basis for a research study for Boston Children’s Hospital that was awarded a 2023 Quirk’s Marketing Research and Insight Excellence Award in the Health Care/Pharmaceutical Research Project category. More recently, she’s collaborated with researchers from MIT and Northwestern University’s Kellogg School of Management on next generation AI. She regularly presents at leading industry conferences such as the Front End of Innovation Continued and The Market Research Event Continued.
Carmel is passionate about the intersection of psychology and business. She holds a Bachelor of Arts in Psychology and Sociology with a Business and Organizations concentration from Cornell University. She also holds an Masters in Business Administration in Marketing and Management from The Wharton School of the University of Pennsylvania, where she was named a Palmer Scholar.
To listen to the interview, search for Product Mastery Now on your favorite podcast player.