A measurement story from Data Stories...

Running your own consultancy is fun when you win business, and less fun when you don’t. Although with a seven-month old at home, the latter can sometimes be a blessing in disguise when you’ve overbooked yourself. Of course, losing the odd pitch or two is all part of the independent consultancy life and not something you should lose too much sleep over. One recent loss in particular has rankled however, if only because it really highlights the same old issues with the measurement of digital ad effectives that a vocal minority have been banging on about for what seems like years now:

• The issues of non-transparent / independent measurement
• A continued obsession with basic behavioural metrics that tell us nothing of real-world impact
• And a focus on measuring short term outcomes rather than long term impact

To cut a long story short, a content creation agency included as part of their pitch to a global advertiser, an overview of how I / Data Stories Consulting would provide some independent market research and analytics to report on campaign effectiveness. Although part of the agency pitch, my contact would have been directly with the client and results would have bypassed any agency filter – i.e. they would have been genuinely independent.

I was delighted for the agency when their pitch won – it was well deserved, but not least because it gave me a new interesting project to sink my teeth in to. It was rather depressing therefore, to be told a couple of weeks later that the advertiser has suddenly decided that they’d like their social agency to handle all campaign measurement. It was no fault of my content agency contacts at all, but rather a reflection of the advertiser’s attitude that social was all.

As means of distribution of course social is vital, but as a means of total campaign effectiveness measurement? I doubt it. As Keller Fay tells us, 90% of brand conversations happen offline. Quite how some social listening tools that are inevitably going to report on the usual spread of clicks, likes and shares are going to report on offline conversations, or even report on total campaign brand impact and long term ROI I don’t know.

I’ve been working with numerous brands and agencies over the past few years to help them better understand how to measure the impact of digital (summary: just like you measure the impact of offline! …kind of), and have produced measurement frameworks like the below to do just that. It’s a straw man and needs adapting for specific client needs, but it’s a useful starting point:

It’s hard not to sounds like a stuck record sometimes, but there’s a reason why the IAB had to launch a national “Don’t be a Clickhead” campaign earlier this year. Clicks, likes and shares might be useful metrics to optimise campaign delivery on, but as a measures of true campaign impact they are virtually meaningless.

Measurement isn’t easy of course, and I hear plenty of excuses as to why it’s not being done properly:

• We get CTR data in real time. How are we supposed to optimise media and creative on brand impact when survey results aren’t available for weeks after a campaign has closed? My usual answer: There are plenty of research vendors who offer real time brand uplift reporting now, enabling you to optimise performance on something far more closely related to real world impact. Attention is also a behavioural metric with some meaning – and there are plenty of scalable measurement options in this space too.
• Measuring brand and sales uplift properly is expensive! Sometimes it is for sure, but arguably a campaign which devotes 5-10% of its budget to measurement rather than media/reach is going to generate far more strategically important learnings for the business in the long run.
• We can’t link to any real world sales data: Again, there are big research and data vendors in this space who would say otherwise. Where sales data isn’t directly available, then there are a world of useful proxies you can lean on instead: for example, efficiency scores which allow you to assess the cost of each incremental purchase intender, or the use of geo-location data to look at store footfall uplifts.

The much publicised Ebiquity and Radiocentre Re-Evaluating Media report last year showed us the huge disparity between perception and reality when it comes to providing independent third party measurement. For example, agencies and brands rank digital video top for transparent measurement, whereas the evidence suggests that quite the opposite is true:

It’s one data point, but it certainly gives a hint as to why we must continue to bang the drum when it comes to reporting on digital impact properly. It’s difficult (sometimes), but that’s no excuse for not giving it a go. If you only get half way there you’re going to be doing your business many more favours than if you just regress to what’s easiest!

Founded by Ian Gibbs, Data Stories is an independent data consultancy focused on all things digital, advertising, media and publishing related. Drawing on over fifteen years of experience in market research, data analytics, digital advertising and strategic planning, Data Stories employs a broad data-driven skill set to help clients solve their business problems. Find out what clients have said about Data Stories here.