Research and the Need to Ask the Right Questions
As a researcher, I vividly understand both the value and challenges of research. Ask too many questions, and response rate plummets. Ask too few, and it's easy to miss out on key insights and come to the wrong conclusion.
Today, I was asked a research question on Facebook and I gave an honest answer. The problem is that my honest answer will lead to exactly the wrong conclusion. The issue is that while asking one question on Facebook is an effective way to maximize response rate, the single-question approach cannot capture all relevant data.
The question was: "Do you recall seeing an ad for <Car brand and with a campaign that involves snow melting> online or on a mobile device in the last 2 days?" (I redacted the car brand name to protect the innocent). My answer was yes, which means my data could contribute to a marketing analysis suggesting that this ad works. It has "high recall," some PowerPoint deck will undoubtedly proclaim.
The problem is that the research approach failed to ask why I recalled the ad. The answer is that I HATE IT. It's almost September and kids are already back to school, and an ad that shows autos racing around causing snow to melt is, in my mind, silly. It makes me think less of the brand, as if its marketers had a great idea last winter and either failed to consider a mid-summer companion ad or ran out of the budget to produce a new ad.
Moreover, even in spring, I did not like that ad. Why would this brand's cars make snow melt? Global warming? Thanks for reminding me how cars are destroying the planet, auto brand!
Of course, I'm just a focus group of one. You may have a completely different opinion of the ad, but my original point still stands: What the research question cannot capture, because the brand did not ask, is why I (or any other respondent) recalled the ad. In my case--and I may be alone on this, I understand--my recall of that ad makes me LESS likely to purchase a car from this brand. And that is an insight that will be missing from the report some marketing analytics expert presents to the CMO.
Human2Human approach to book sales calls and fill your pipeline via LinkedIn. No pushy tactics, no cold calling, #nobots. CEO, PropelGrowth
7 年I wonder if they would have thought it a positive response regardless, just because you recalled the ad so clearly? If their goal is awareness, could a short-sighted agency conclude that they'd succeeded even if they had the additional insight? Not saying it's right, just wondering what you think.
Data>Information>Insight>Knowledge
7 年Your post Augie certainly makes sense. To completely torpedo the example you use one should have begun with the point that "aided recall" is as gross (both interpretations apply) a metric as one can use and no serious experienced researcher would ever use it - particularly as you say without supporting qualitative playback. The problem today, more so then ever before is that everyone believes themselves to be a researcher.
Proven Senior Marketing & Branding Executive | Helping Companies Translate Their Business Goals to Reality
7 年Speaking as a career marketer, your point of it showing up in a deck labeled as "high recall" really struck home with me. Over the last 20 years I've developed an extremely low tolerance for "bad marketing/advertising"...and the fact that most marketers today accept research findings too willingly. I can't remember the last time I heard a deep and challenging discussion around the methodology.
Corporate Positioning, Branding and Marketing Communications Professional
7 年Very true assessment Augie, and there in lies the fallacy of using digital devices or platforms for purposes of conducting research: the maximum attention span is pegged somewhere around 8 seconds for mobile devices. In addition to asking the right questions, one has to ask them of the right audience. Thanks for sharing your experience.