Creative Testing Methods

In Digital Marketing, I’ve learned, there are many ways to get results. Testing creative is one hot potato where companies, agencies & bosses have their ‘method’ they subscribe to. We’re going to deep dive into MVT vs MCT vs AB testing.

Firstly what is MVT/MCT testing?

Multi Creative Testing is where you can upload and use a vast array of creative/ads ato test, or use a campaign format to allow the platforms (Google, LinkedIn, Meta Ads etc) to optimise to the best performer (be careful here, more on this later.). Essentially isolating the creative as the only variable to test, to see which works best.

MCT vs MVT

This is the difference:

MCT - testing (only)? multiple creatives at the same time

MVT - testing multiple variables such as creative and placement, at the same time

How do you do this? DCO helps...

With Facebook/Instagram/Meta Ads you can create campaigns with multiple creative and select Dynamic Creative Optimisation for the campaigns to be optimised according to the best performing creative. You can also test creative on Google Ads, Display and Youtube, and on plenty of other ad platforms.

Re DCO, it can work well, although complicates the matter a bit with reporting, when brands turn on multiple placements. The reports and requisite pivots can get a bit tricky, especially when comparing multi time periods. That said opting for DCO opens you up to testing a lot of creative in a short space of time, and cuts down on campaign builds too (yay). Though the auditing, reporting and finding insights is more intricate and requires someone analytical, with a keen eye.

How about Text Ads/PPC?

Google doesn’t skip out on this. RSAs stand for Responsive Search Ads, and allows Google to mix and match multiple headlines to, again, optimise to the best performing creative. For this article, I’m focusing more on the creative element, so will leave out Text ads, but the principles are similar. *

*Actually PMax campaigns are sort of pumping out dynamic creative anyway

Why would a brand or agency want to opt for mass producing tons of creative?

It sounds like a lot of work. And yes, it is. Typically BOTH larger firms and smaller start ups do not like the mass production model of creating boat-loads of creative to ‘just test’. Most brands, and many people, prefer a more docile A/B test, and doubling down iteratively. I don’t subscribe to either view, as you’ll see. When done well, they both get good results.

I want to do MCT and Test bucket loads of Creative! Where/How can I?

Creative agencies. Of course! They have the means and personnel to create a bunch of creative. And the skillset. In my experience (after being initially baffled), the volume of creative can be astounding, but after auditing the campaigns (my role), one really can find some very very interesting, statistically significant trends from running these ‘mass’ creative tests.

Alternatively internally you could structure a ‘performance creative’ function whereby a creative person is dedicated to creating variations of creative. Caveat here is that creative agencies use talent pools across disciplines and can typically produce a wider variety, including photo and video shoots and animations, which are harder to replicate like for like when doing things internally, especially for smaller companies.

Other pros..?

One of the main pros of running mass creative is you’re stripping out a big variable that is hard to replace. Time.?

An example explains aptly.

Imagine Brand A decides to enlist a creative agency to produce 100 creative per quarter to test for a year. Versus, Brand B that uses there in house team and does a few A/B tests. At the end of the year they both audit the results. They both draw conclusions, then double down on the things that worked. Now imagine the first year is 2019, the second 2020.

Brand A (mass testers) will have tested 400 creative and will know what works in 2019.

Brand B may have only tested a small fraction, so may know some broad strokes.

In 2020 they reinvest according to their knowledge of results and Brand A & Brand B do well, until Q2 when Covid lockdown the world.

The Cons

I know what you’re thinking. And you’re right. Mass production of creative is both hard and for many people, not so desirable. It’s a real operation ( but can yield good results).

The other con is if you do use a lot of creative, and you open up placements on FB/Insta ( or other future/ different platforms that have multi placements) you are reducing your budgets divided by the (often unequal) splitting of the traffic by placement. Meaning? Often you don’t get good enough click/lead/customer volumes to make good (significant/valid) decisions, as the volumes can be low (spread too thinly across creative and placements). 1000 clicks over 2 placements is 500 clicks, over 10 placements it is only 100 clicks. If one gets a low conversion rate, many placements, in this scenario might have 0 conversions, while others may have more than one, but the low volume doesn’t speak to which creative or placement actually will perform when you scale out ( add budget) to these campaigns/creative. So be careful.

Not every year will be 2020.

You’re right. And when one tests large differentials within creative, A/B tests can indeed produce valuable insights. You can decide what a minor or major variable within creative means to you and your brand. Context is, as ever, king (or queen).

Where else have I seen diametrically opposite views on ‘how’ to run campaigns?

On PPC there are some that build broad campaigns first, and others that expand carefully, only when they see meaningful results. This is similar to the creative expansion model. It’s not related to Creative, but still interesting, in my book.

Small(er) budget Options

I’ll be honest, hiring a creative agency to test out 100 variations or more costs bank. If you’re working with a smaller firm or client, you can still run creative tests using DCO on FB & Insta, just scale up and down the volume of assets you produce based on your budgets.

Middle Ground

Testing less creative over less placements, to get to statistical significance/results faster. That way you can double down later. Requires someone to analytical analyse the data as well as the ‘in creative’ variables such as camera angle, colours, copy etc

Micro Budget DCO - Cheapest Creative Testing

What’s the cheapest way to test Creative, I hear you ask? Display. Using the Google Display Network you can easily and cheaply test Creative, often for a 1p a click. You can also test by country, choosing a lower CPC country to test creative out, before embarking on a more expensive country. Why waste budget when you’re looking at trends and patterns of what creative leads to engagement (CTR) or lead vol at a low cost (CPA)? Exactly. Think Display.

But Jake, I want to Test my Brand Videos!?

If you’re testing Video Creative, YouTube ads (using bid caps at 0.01) allow a cheap way to test video creative. Why Brand Managers don’t use this before splurging millions on TV Ads, I’ve been perplexed by since 2012.?

I’ve always been keen to marry Performance Marketing to Brand, shame the world/brands do not always see things my way. Sighs… Maybe one day when I’m a CMO or CGO (Growth), I’ll be able to throw my hat into the ring on that one..

Real World Example - Statics

Tim Ferris tested his book covers (himself - every good marketer tests themselves), on Display for ‘The 4 hour Work Week’. He also split tested the title. Fun Fact.

? ?????????????????? to this newsletter if you like the content, keeps me motivated.

? Super ?????? ???????????? to the over 350+ subscribers I've got already

? Let me know your ???????????????? in the ????????????????, thanks!