Data Detective: The Case of the Blockbuster That Wasn’t w/ Andy Hasselwander
The below article contains show notes from Episode 2 of The Last Mile Podcast. Rather listen than read? Click here.
Here’s a detective story about understanding what happens once a product or service gets in the hands of real customers.
Andy Hasselwander, Chief Analytics Officer at Marketbridge, used the science of ethnography to revive a product that should’ve been a blockbuster, but just went bust instead.
The Case of the Missing Blockbuster
“When you’re seeing slow sales on a product that’s supposed to be a blockbuster, don’t assume it’s just one thing. It’s probably many things.” – Andy Hasselwander
Hasselwander thinks of himself as an applied analytics executive — less a data scientist, and more of a consulting detective.
“I spend most of my time trying to figure out how to use information from the market, from customers,” says Hasselwander. “And I use that to solve go to market problems.”
There was one case Hasselwander solved that dealt with the three major challenges companies face today in terms of revenue growth:
- Customer adoption
- Customer retention
- Customer engagement
There are some who believe that 80% of your revenue, if not more, comes from retention and cross-sell. So keeping the customers happy is mission critical.
In this case, the clients had created a pretty new product that was solving a need that had previously been resolved using primitive technology.
“That client was having two problems. The first problem was that their uptake, as far as new product adoption, was substantially lower than they’d expected. They were anticipating a blockbuster product,” recalls Hasselwander. “Then once that product was installed [in offices of various sizes] they were having problems with retention. In this case, retention meant ordering additional supplies and continuing on a service contract.”
The product worked as advertised. Everyone agreed it did the job better than all previous methods, and yet there had to be some clue about why it was underperforming.
“I immediately started seeing evidence of friction. I was hearing a lot of little anecdotes from salespeople, from channel marketers, from analytics, from researchers about stories they’d heard,” says Hasselwander. “It’s usually it’s the small things that are the cause.”
“Identifying hurdles, speed bumps, points of friction and prioritizing those in knocking those off one by one, that’s how you get to really great.”
Hasselwander did a deep dive into the data to discover that this product had:
- A channel distribution problem.
- A hidden influencer group that was negatively affecting adoption.
- A serious UX problem that was making it hard to buy and harder to renew.
“Those three together took what should have been a 20% year over year growth, and turned it into an anemic life support product,” says Hasselwander.
Ethnographic field observation to the rescue
Analytics data can be used as hypothesis generators, but truly solving these problems required fieldwork. Hasselwander deployed researchers to do ethnographic research on the company’s customers.
Wikipedia defines ethnography as “the systematic study of people and cultures. It is designed to explore cultural phenomena where the researcher observes society from the point of view of the subject of the study.”
“The goal of ethnography is to not be noticed. The goal is to observe,” says Hasslewander. ”You ask almost no questions. You need to go in with a small team and just watch.”
Hasselwander divided 60 companies into two groups of 30. One was companies that were using the product. And then the second was companies that were in the prime prospect definition but were not using the product.
After 45 minutes or so of watching the two groups interacting with the product, the research team would ask a few questions, and then ask them to pretend to renew the product (or buy the product for the half of the group that wasn’t already customers).
Doing this uncovered massive problems with user experience. Both renewal and purchasing revealed that UX was very hard to navigate and required setting up two-factor authentication which caused serious cart abandonment issues. People became confused and frustrated.
There was also a previously undiscovered emotional component to current users’ lack of engagement. Previously, everyone had had to interact with the office manager to get their supplies ordered. Now the process was automated, so the office manager became increasingly lonely. Even though it was better for everyone in the company, the office managers had a real resistance to using it because it made their jobs less enjoyable.
“People seek emotional connection in their jobs. We found that this product was boring, and sort of sucking the soul out of these people,” says Hasselwander.
Observation revealed a search optimization issue. People couldn’t find the website easily doing regular term searches for the site. The engineers assumed that everyone would bookmark the appropriate page, but few people did.
“We immediately noticed we had to use all the search terms to get them to the right place, and you had to click through four different screens,” says Hasselwander.
Observation also revealed that the IT manager was a major influence on product adoption. This required a revamped two-audience marketing strategy instead of just targeting the office managers.
The Solution
Using an agile development methodology, Hasselwander created self-managing pods to address each of the issues that they had unearthed.
To continue reading click here or listen to the podcast episode on iTunes.