Train Drivers Laughing at Salad

Many years ago, I had an ad agency client phone me up. They were not having A Good Day.

Apparently, our lovingly-crafted golden questions algorithm for their shiny new segmentation was wrong. This had been sent out to thousands of consumers already.

Shit. I had that horrible sinking feeling you get when you know you’ve screwed up. You know the one. That feeling you get when someone says “I’m not angry, just disappointed”.

I asked how they knew it was wrong.

Apparently the algorithm had allocated someone into their target segment. A group of people who love their home, their appearance, are self-confident, and want to make the best impression of themselves onto others. This person ticked all the right boxes.

But – despite all that – it turns out THEY WERE A TRAIN DRIVER.

??

As politely as I could, I suggested that perhaps train drivers were allowed to feel pride in their home & appearance, and that real consumers were not Sex-In-The-City-style airbrushed caricatures of themselves.

I checked the algorithm. It was fine, of course.

Scroll forward a few weeks, and I saw a presentation of the same segment, using 2 videos.

One set was created by the ad agency – all found footage of people rollerblading and women alone laughing at salad.

The other set was from in-home ethnographic footage. And, to continue the meme theme, some of the footage looked like something off Terrible Real Estate Agent Photos.

The look on the ad agency client’s face was a sight to behold.

Anyway, what’s my point?

1. Segment Personas are almost always caricatures of the people they are supposed to represent. And we insights professionals must always tread a fine line between playing up to those caricatures, to help bring definition and clarity to the segment profiles, and being the “data police” – making sure our natural tendencies to stereotype are bounded by what the numbers are telling us.
2. It’s absolutely fine to have a ‘bullseye consumer’ in your thoughts when creating campaigns or products – but remember that 99.9% of people are not caricatures. Which leads me on to my final point…
3. ...real people are complicated. Inconsistent. Contradictory. Highly dependent on context and mood. I put people into buckets for a living, but I never see these buckets as completely static, distinct groups. They have fuzzy edges which are always moving.

And I say: as researchers, we should lean in to this complexity. Segmentation is a tool to help bring a clarity of purpose to marketing efforts. What it is not is a tool to help people dumb down people’s real lives.

And we all know this. But how to do this without confusing your audience? A few suggestions:

• It's often useful to start by showing the ways in which consumers are similar, rather than how they are different. What are the table stakes for the category? What elements don’t differentiate? This helps frame the context when discussing the differences.
• If you are going to use personas, I always recommend having multiple versions to represent each segment, to avoid stereotyping them as “you, know the person wearing the Teenage Fanclub t-shirt; the one with cats”. And please, for the love of all that is holy, don’t call your segments names, like “Rod, “Jane” or “Freddy”.
• Focus on the original reasons for doing the segmentation, rather than the segmentation itself. Build it into specific business processes. Engage with different stakeholder groups, and give them tailored deliverables.

One final point, and it concerns the latest way that Market Research has decided it will eat itself - synthetic data.

I’ve seen some incredible examples of what synthetic data can bring to the table recently – and I’m using it in a couple of projects at the moment, where traditional quant isn’t feasible. But I’ve never really had a good answer to my main question about using synthetic data as a more consistent replacement for real respondent data: how can LLMs recreate the inconsistent & contradictory nature of real human beings?

Frankly, every synthetic data demo I have seen so far has been much more like an ad agency video than an ethno video. This airbrushed version of reality might pass the sniff test with flying colours, and might make as much (perhaps more?!) sense in aggregate, but I surely it falls down when looking at individual respondents, and doing anything like cluster analysis.

Yes, I know some of this inconsistency can be down to respondent fatigue, or poor questionnaire?design, or myriad other traditional criticisms of research. But it’s not all down to that. And I think that by design, synthetic data leans too heavily on caricatures. It’s no coincidence that pretty much every tool has a ‘Personas’ option.

Anyway. Would love to hear people’s thoughts on this!

• What are the best ways you’ve found to tell a story around segments without reducing it to caricature?
• What are people’s views on Personas generally?
• And how can synthetic data imitate the inherent complexity of real people?