That's only half the game...

How to test direct mail campaigns Richard V. Benson and Craig Simpson of Simpson Direct, Inc.

Legendary direct mail guru Richard V. Benson was always looking to improve the outcome of his own campaigns, and help other marketers do the same.

While he agreed that creativity is important in preparing great mail packages, that’s only half the game. The other half is analysis.

He claimed that the failure of many campaigns could be attributed to the fact that their creators ignored or misinterpreted their results. In other words, they failed at the analysis stage.

So, in this email we’re going to look at two elements that Benson felt were essential to good analysis: bogies, and testing.

Bogies

In order for a campaign to be successful, the cost of acquiring each new buyer must not be greater than the income from the sales made to that buyer – otherwise you’re losing money.

According to Benson, the critical figure the advertiser must keep in mind is what he called the “bogey”: the maximum allowable cost for acquiring a new customer. Calculating the bogey tells you how large a mailing you can send out and make a profit.

The method Benson offered for calculating bogeys is good for any program or product that has the possibility of a second sale – that is, a program where each new customer can keep building lifetime value after the first sale.

Benson believed that if the advertiser broke even on the second sale to the customer, the campaign could be considered a success, with no question that profits would soon outweigh the cost of acquisition as purchases continued.

Basically, Benson would determine all the costs of the first promotion, including fulfillment costs. Then he would determine the income from the first and second sales. From this he calculated the level of response needed to break even.

For example, let’s say the net income from the first and second sales of a magazine subscription is $14.42 per customer, and the direct-mail costs to acquire customers are $240 per thousand.

To make a profit you must have an order response of 1.66% to break even (16.6 responders per thousand mailed x $14.42 profit from each responder = $239.37 income per thousand). On the third sale you will make a profit.

?

As you can see, lifetime value per customer is critical for this analysis. You always must consider lifetime value when determining the profitability of an advertising campaign. (If you’re not sure how lifetime value works, call George Platt at 203-640-0421.

Then Benson juggled the figures a bit more. If you plan to rent out your list, that would add to your income from each new customer.

Adding that into the calculations, you might be able to make a profit with only a 1.46% response, allowing you to increase the number of mailings sent out by as much as 30-50% - leading to more orders and more profitability.

This is how you approach your mailings like a real businessperson. You know exactly how many orders you need to make a profit, and how to assess the outcome of every mailing you send out.

Running Tests

Benson was a major proponent of testing direct mail promotions. He said testing was “the be-all and end-all of direct mail marketing.”

He believed it was worth testing any factor that you believed could increase response: “The biggest failure in the practice of direct mail is the preponderance of preconceived mind-sets.” So don’t assume anything about what will work and what won’t work. Make the effort to test it out.

First, Benson thought that concept testing was mandatory for any new product. That is, before launching a product, you should survey potential buyers and ask what their likelihood is of buying it. He suggested providing four to six options for prospects to choose from (kind of like a line up in a police station).

For controls he presented two existing products or past products, one of them a winner, and one a loser. Then there would be a number of other options, including the new product you’re considering launching.

Once a product has passed the concept testing stage, he would run a dry test. This is where you send out a mailer on a product that is “not yet available.” You’re still not sure you want to go ahead and launch the product. You’re trying to find out if people are going to buy it before going to the cost and expense of creating it.

This is not something a small operation would do. According Benson’s plan you would need to send out a 150,000 piece mailing with different combinations of lists, offers, sales packages, etc.

More specifically, he thought the initial testing to determine the viability of a new product should include tests of two to three mail packages (created by different people!), two to four different offers, and up to 20 different lists.

I’m not suggesting you would do this. I’m really telling you this so you can get an idea of how the “big boys” at major advertising agencies approach a product launch. It’s pretty impressive.

Analyzing the Results

A lot of Benson’s advice is applicable to all of us. First, Benson cautions that when looking at results, be sure to take lifetime value into consideration. Without considering the long-term sales to a customer, it’s possible to consider a mailing to be a flop, when in fact it was very successful.

Benson also said to be very careful in analyzing test outcomes and be very wary of illogical results. Some of these may be due to not having enough money to put together a perfect campaign, and suspect outcomes can be the result of running tests that are too small.

Plus, there can be chance differences across groups that could account for up to 15% difference in results. Just looking at statistical outcomes doesn’t always give you the whole picture. As Benson put it, “Too many people hide behind the computer and are afraid to take risks.”

And Benson was a bit of a maverick in other ways. For example, most statisticians will tell you the only way to do proper testing is to vary just one element at a time. Benson said that statistically that’s irrefutable, but “there’s never enough test money available to test every ingredient.”

He believed the best way to end up with a winning mailing was to test total creative packages against one another, while keeping the lists and the offers constant. He said, “The goal is success, not statistical purity in original testing.”

In my mind, he is probably correct for the huge mail packages he tested for wealthy clients with national products. But for the kind of businessperson you probably are, with much smaller mailings, a simple test where groups are properly divided with up to 5,000 names per cell, where only a few parameters are tested, and where it’s possible to tease out the results of each parameter separately, will work best.

#marketing #directmail #innovation