"What Works?" is a difficult Q to answer

It’s (always) a good time to double down on customer retention.

The trouble is, while the ROI is evidently high, it’s sometimes hard to figure whether you’re doing the best you could. The question that comes up often is – what kinds of campaign mechanics are doing well. What should we do more of? What should we do less of? And this, I’ve often found, is not an easy question to answer. Here’s why:

Fearlessly Try Stuff

If you really want to figure what works, you need to have tried a whole lot of things without fear of failure. Does reaching out to small sets of customers with targeted messages work better than mass outreach with offers? Can’t say till you’ve tried (no it’s never as straightforward as “Yes, targeted works better!”). Does personalization actually help? Does sending on Friday work better than on Wednesdays? Does a 50% Off work better that BOGO? Many organizations just haven’t tried enough to be able to get good insights so the point is - keep trying stuff. If you haven’t been doing it already, start now. If all you’re doing is mass campaigns with an offer and no personalization you aren’t going to be able to optimize campaigns.

?

Benchmarks from “out there”

Ever so often I get asked to help with “what’s the benchmark”, how well are we doing compared to our peers. The only real answer, I’m afraid, is: You are your own peer. Do better than you did, and keep optimizing. Every firm operates in a different context. Even if they look similar, maybe their loyalty program enrollment criteria is different, maybe the number of outreaches per month they do is different, so many maybes... So don’t bother with what’s being curated and put out there in conferences and case studies, just keep doing better than your own past self.

Why A/B Testing isn’t the (complete) answer

This line of discussion often gets around to A/B testing. Nothing against it, but to me you can’t have a winner takes it all strategy when it comes to campaign mechanics. A/B testing is winner takes it all – you see which option worked better and then divert all weight there. Variant A has a % Off offer, Variant B has an Upgrade offer – if A does better than B, it doesn’t mean all comms should now go out with the % Off offer. It does however mean one should amplify the winner strategy and suppress the loser – till perhaps things change. And that’s the key point – things change, context changes, people change – so what they respond to will change, and if you shut the door on an option, you miss picking up a signal that could be vital.

Enter MABs

Multi Armed Bandits are fascinating. The name originates from a Casino-beating strategy wherein you put some tokens in multiple slot machines and crank their arms (a Slot Machine = one armed bandit, stealer of your tokens. Many slot machines = Multi Armed… you get the drift). You feed more tokens in the winning machine(s) and less in the losing ones, without diverting all your money to the winning one. This is where ML takes traditional A/B testing and amps it up – exploit winning campaign mechanics by diverting more comms there, suppress losing campaign mechanics but don’t shut the door on them.

Sometimes, all you want is to know what works

This is where we started, right? MABs can solve for a lot, but they’re not great at actually spelling out what works and what doesn’t to you. Good old descriptive analytics does that just fine. In SOLUS we just released a new module to help analyze what campaign attributes work. Here’s how we’ve done it:

Campaign Intelligence in SOLUS

• We’ve broken down all campaigns and triggers into attributes like Type/ Segment/ Timing/ Size/ Channel/ use of Personalization/ Offer type/ use of Recommendations etc
• We measure them on Metrics of Yield (Incremental Rev per Outreach), Conversion %, Incremental Revenue $
• We give you the ability to look at variables in isolation or combination

Couple of no-context screen-grabs below: You have to see these on your own data, hence no attempt to describe these charts:

One more where you see campaign attributes in combination:

This new module in SOLUS is incredibly useful. Give it a look (or try), and if you’d like to know more about how we work with MABs to auto-optimize campaigns, let me know!

Cheers!

Sandeep

?www.solus.ai