Recast

The marketing talk of the town of the last 5 years has been all things omnichannel. But how can marketing budget owners make spend decisions that are best for the business when distro ends points are fragmented and signal is so noisy? I have thoughts, now up in my first ever Recast blog post! :) Content Warning: This may lead to you flexing your measurement knowledge and potentially driving more meaningful business outcomes. Not for the faint of heart! *** Happy Friday, all!

Many Recast customers have tried open-source media mix modeling (MMM) tools before deciding to use Recast. While these packages are interesting, they only solve the easiest part of MMM: running the model and producing some output The real challenge lies in operationalizing those results. How do you validate the model? Communicate the insights to marketers? Get them to trust and actually use the findings? That's where the open-source tools often fall short without substantial engineering investment. We've heard from organizations whose data science teams run MMMs, but the marketing team is completely unaware of or disengaged from the results. If marketers aren't leveraging the insights, what's the point? Customers turn to Recast because they recognize the potential of MMM but need help making it actionable. They need a solution that ensures consistency, stability, and trust in the results. Pushing data through an algorithm is easy. Operationalizing the insights is the real challenge.

How marketers measured performance in the Mad Man era doesn’t make sense today. Media mix modeling has evolved and we are able to do so much more now than what was possible in the past. It’s easy to imagine why MMM was so important 30+ years ago: Let’s say you were the CMO of a large CPG brand back in the 80s. The world looks pretty simple: there’s no e-commerce or any way to track people. You’re advertising via radio, print, TV, and in-store promotions. Then, after the ads have been run you run a statistical model to see if there are any relationships between the advertising spend and in-store sales. This was usually done on aggregated data (e.g., monthly sales) and the results were more directional than precise. Back then, there were a few consulting firms that, every six months, would go to the CMOs and present these analyses so the brand could make a budget and go buy media (like in TV upfronts) for the next six months. But things are different now, and that means we need different approaches to doing media mix modeling. There are 2 main changes: 1 - The way we buy media is much more dynamic. Instead of buying most media at upfronts, today most media is bought via dynamic digital auctions. Media spend decisions are happening every day, every week, every month, not aligned to an annual planning process. MMMs need to be sped up to match this faster decision-making cadence. 2 - Technology has improved. With modern methods, we can estimate much more complex models accounting for more of the nuances of how marketing works in the real world. Altogether, that means that we should be estimating more complex models, faster. And that’s exactly what we’ve built at Recast.

If you are a small business present in only one or two marketing channels, the number of metrics you need to look at is going to be different than if you were at a multinational, multi-brand, omnichannel type of retailer. That being said, thinking there is one single source of truth for marketing measurement can be misleading and even hurt you more than actually help you. Here's why:

If you're going to do an MMM, you should think a lot about how you are going to validate the model. How are you going to build trust in it? How are you going to know that it's right? The reality is that, with a flexible modeling methodology, there are often millions of ways that the model can go wrong and only one way that it can go right. There are a few different ways we think about validating MMMs at Recast:

In marketing, the key to success is selecting analytic tools that align with your business stage. Whether you're a startup or a multinational, your analytics strategy must evolve to match your scale and complexity. Here’s how I think about it:

Lift tests are a great way to calibrate your MMM – but it has to be done well. You want your tests to give the MMM more information about the ground truth, which will help the model settle on the set of plausible parameters that are consistent with the results that you got from the test. This can be tricky because the vast majority of MMMs make the assumption that marketing performance doesn't change over time. If you have two different lift tests with two inconsistent results from one channel, which is very common because channel performance changes over time, it's not clear which of those lift tests you should use when you're calibrating your MMM results. What we do at Recast is, since we have a Bayesian time series model, we're estimating what the incrementality of every marketing channel is – for every day. That lines up really well with the way lift tests work because we can incorporate those results directly into the Bayesian statistical model by putting priors on the performance of that channel, but only at the time when the lift test was run. We're treating the evidence correctly by considering it a snapshot in time and not applying it to how that channel has performed over all of history. MMM and conversion lift studies don’t operate in silos, and we highly recommend using them together to get a more clear picture.

Doubts about an internal MTA tool, expansion into brick-and-mortar stores, and a renewed focus on efficiency… The Daily Harvest team had a lot on their plates when they first came to Recast. Over the last two years, Recast has helped Daily Harvest resolve many of these measurement challenges. It’s also become their go-to platform for marketing planning and driving efficient new subscriber growth. If you want to read more about Daily Harvest’s measurement journey with Recast, check out our new case study - the link is in the comments!

As a media mix modeling vendor, we very often tell companies not to work with us because MMM isn’t a fit for them (yet). It’s usually because of the same three reasons: → Limited Channel Mix: If your channel mix is just one or two channels (Meta and Google, for example), in-platform reporting and lift tests might be enough and you don’t need to overcomplicate your measurement strategy just yet. → Low Volume of Marketing Spend: With marketing spend lower than $5M per year, there are often higher ROI activities than investing in MMM. MMM is really complicated and there’s a lot of internal organizational work you need to do to make it successful. If you don’t have that much marketing spend, the benefits of optimization are lower and so the ROI of an MMM project is itself lower. → Lack of Organizational Willingness to Act on MMM Insights Believe it or not, it's not uncommon for companies to do MMM, get some valuable insights, and be unable or unwilling to implement the recommended changes. If your marketing team is not ready to pivot strategies, reallocate budgets, or experiment with new channels based on MMM insights, then just don’t do it at all and save yourself the effort. Now, if you’ve read the list above and realized that your organization might not be ready to implement MMM just yet, that’s okay! You can focus on building the foundation to help you prepare for it once you’re ready. Start educating your team on why incrementality matters and focus on building your “experimentation muscle.” After $100k+ / month in paid media budget, it’s a good time to start building that culture with geo-lift and holdout tests, for example. As testing, data, and incrementality become part of your company’s DNA, the transition to MMM will feel like a natural progression rather than a disruptive shift.

What does a good holdout forecast accuracy system look like? Here’s how we think about it at Recast: