Marketing Isn't Measurable

Yes, the title of this post is a bit clickbait-y, but it’s a statement that I believe to be true to a certain degree. To be more precise, I believe that specific aspects of marketing cannot be measured with 100% accuracy or certainty as programs scale in complexity.

For example; if a newer business generates a negligible amount of organic traffic, and the business’s only real marketing effort is a direct response Facebook campaign, then measurement becomes simple. The marketing team can rely on Facebook’s in-platform measurement for performance reporting with a high degree of confidence. With little traffic coming from other sources, it’s likely that any incremental customer volume is coming from the Facebook campaign.

While this dynamic might describe organizations that are just starting to build an online presence, measurement is unfortunately much more complicated for more established brands.

If a business has substantial organic traffic going to its website, and is running campaigns across both Facebook and Google search, measurement becomes more complicated. We can run through two scenarios to map out what different levels of complexity look like.

Scenario 1

A customer’s first interaction with a brand takes place on Facebook. The customer clicks on the ad, but isn’t ready to purchase so they leave the landing page and conduct a bit of research on the product. Through a Google search the customer clicks on a paid search ad, and purchases.

The first issue in this scenario would be relying on in-platform reporting. Both Facebook and Google will use different attribution models here, and don’t have visibility into the brand interactions the customer had outside of the separate ad ecosystems. Facebook might report that the ad interaction on its platform accounted for 0.5 purchases, but Google might report that its paid search ad can be credited for 1.0 purchase. Since we are lucky enough have an omniscient level of context in this example, we know that the customer only completed 1.0 purchase, but using data from both platforms would indicate that 1.5 purchases were actually completed.

Once upon a time this attribution issue could have been solved with a tool like Campaign Manager 360 and floodlight tags, but the effectiveness of third party cookies continues to wane as we move closer to cookie deprecation. To throw another wrinkle into this scenario, when that customer turned to Google they actually conducted a branded search. How can we determine if that paid search ad really captured incremental value when the brand ranked in position one for that search organically?

We can see here that moving from just one traffic source to three makes measurement significantly more difficult. Before diving into potential solutions, let’s explore another scenario.

Scenario 2

A much more established business has a large amount of organic traffic coming from search and social, and the marketing department is running paid campaigns across search, social, programmatic display, and connected TV. Additionally, the brand purchased some more traditional out of home inventory and has a strong level of brand recognition.

A customer could have dozens of interactions with this brand before completing a purchase. Trying to attribute credit to digital channels here will be difficult enough as it is, but how can the company determine how impactful their billboard ads have been in its media mix? What about word of mouth gained by strong brand recognition?

Was there one interaction that stood out for this customer, then they started to notice the brand everywhere (Baader Meinhof Phenomenon)? If so, what interaction resonated with the customer the most to drive this effect?

As you can see, the more complex a marketing program becomes, the more difficult it gets to accurately determine what is driving revenue with confidence. Some teams might be left with more questions than answers.

Potential Solutions

In the current state of digital marketing, there isn’t a single methodology or tool that you can plug into your program to answer some of these difficult measurement questions. However, a combination of different approaches can help you and your team to gain a better understanding of what’s driving performance.

To understand the effectiveness of individual channels at a high level, a solution like media mix modeling (MMM) can be effective when analyzing large data sets over a long period of time. Media mix modeling using statistical modeling to measure the impact that different marketing channels have on overall performance. For example, if you were to expand your media mix to include paid social, an MMM tool would be helpful to provide insights regarding that channel’s impact on revenue over a few months. However, one of the limitations to MMM is the level of granularity with which it can provide insights. Getting campaign or creative level impact can be difficult.

To address a more granular level of impact, methodologies like incrementality tests can shed some light on performance within a shorter time period. A common incrementality test method is using location testing. To be brief, location based incrementality testing is basically identifying a marketing activity to be tested, withholding that activity in location A, and implementing it in location B. From there, you can compare performance between location A and B to understand the incremental lift that the tested marketing activity provided on your chosen KPI.? A potential drawback of incrementality testing is that a limited number of variables can be tested at any given time, and the length of these tests might not be long enough to account for marketing efforts with a longer time horizon such as building brand awareness.

Incrementality testing and MMM are two approaches that many more advanced organizations use in tandem to paint a bigger picture of overall marketing effectiveness. It can be helpful to shift your mindset on measurement if you’re part of an organization with a more complex marketing program. It’s unlikely that your team will nail down exact attribution and impact at the creative level over long periods of time. To some degree, all measurement is directional.

Building out a measurement framework that provides you with enough insight to confidently make decisions is what is most important.

For example, you can measure the overall impact of marketing using something like the marketing efficiency ratio to provide a quick gut check, then break that down a level with MMM to understand what channels drive results over longer periods of time, and take your measurement approach even deeper with incrementality tests to understand the short term impact of more granular tactics.

To wrap things up, not all marketing is measurable, and that’s ok. There are tools and methodologies that you can put in place to get a good directional understanding of what activities are driving your bottom line, and continue to iterate and test from there.