Improve Marketing ROI with Experimental Design
Consumers are bombarded with thousands of marketing advertisements every day.
Messages delivered by different media, from traditional TV or telephone spots, through supermarket flyers, to banner ads on websites and newsletters.
These external stimuli can trigger a response from consumers: buying a product, clicking on a link or adding an item to an eCommerce shopping list.
However, most marketing campaigns fail to achieve their goals.
For businesses, it is crucial being able to anticipate consumer behaviour and understand which messages will generate consumer interest, thereby improving the conversion rate (a key parameter in E-commerce or lead generation ) and increasing turnover.
Knowing which ads will actually lead to the target is tricky, as there are different ways of combining marketing campaigns.
That's why experimental design, an essential tool for marketers, is so important.
Experimental design can help marketers analyse how the various components of a campaign influence consumer behaviour.
This approach is significantly more accurate and cost-effective than traditional market testing, as it is possible to match marketing ads directly to consumer needs, and consequently, increase revenues.
Traditional approaches
Testing different forms of adverstising in marketing isn't a new practice.
For example, in direct marketing (a form of targeted advertising directed at one or more users who respond to specific features, with the goal of producing responses and/or transactions), simple techniques such as split mailing (allowing you to create up to five different variants of a campaign to be sent to different parts of your list) are used to compare customer reactions to different prices and promotions.
While this method is effective for e-mail, when trying to evaluate alternatives for marketing campaigns, traditional tests are no longer beneficial.
A traditional method used in e-commerce and marketing campaigns is 'Test and Control'.?
Marketers send advertisements to selected customers, the tests, and compare their results against a randomly selected group of buyers, the controls.
This technique determines campaign effectiveness by comparing sales generated by the 'test group' and the 'control group'.
For example, to test a price, one starts with a control cell indicating a base price and then adds test cells for higher and lower prices.
To be able to test five different prices, six promotions, four banner ad colours and three advertisements, you need one control cell and 360 test cells (5x6x4x3=360).
However, where the complexity increases, it is possible to test more than hundreds of thousands marketing campaigns.
Furthermore, this approach does not reveal which individual variables cause higher, or lower, responses from customers; as the result you get reflects the combined effect of more than just two variables.
You would not be able to determine whether it is the price, or the offer, or the new message that causes a higher conversion rate.
Moreover, in recent years, companies are able to change their marketing campaigns much faster.
Whereas just a few years ago, to change prices and promotions, you had to reprint stickers or distribute flyers, today, organisations can change prices and promotions electronically, simply by reprogramming checkout scanners.
Internet has reduced physical limitations on prices, packaging and advertisements, allowing companies to change product prices and promotions every day.
Being able to change advertising banner colours, customise promotional messages in tone; Internet has provided marketing departments with more and more alternatives than before.
More alternatives, however, means more complexity for marketers to manage.
Understanding this complexity can lead to greater flexibility in marketing plans, effectively meeting consumer demands.
This is why it becomes essential to rely on experimental design.
Experimental Design in Marketing
Experimental design in marketing allows you to predict many ads' impact by testing just a few.
By using mathematical formulae to test subsets of variable combinations, representative of the entire complexity, marketers can create hundreds of campaigns accurately and efficiently.
Experimental design makes it possible to define and control variables before they are even placed on the market, therefore testing different ads on customers instead of observing them once they are published.
It enables you to test how variables, such as price, promotion, or colour, led to a specific customer behaviour, instead of simply being a consequence of a customer attitude.
Even though experimental design is not new, few marketing departments have started to use it, but new technologies are making experimental design more accessible and easier to handle.
Companies are therefore able to gather information more accurately, using the data to construct ads that can trigger a consumer response, with greater speed and accuracy.
Using experimental design requires a good knowledge of the target market and a high level of expertise in mathematics and statistics, and is particularly useful for companies that have multiple customers and have to deal with rapid and continuous change.
For example, in e-commerce, where customers tend to change their demand frequently.?
In fact, attracting new users to one's website and then converting them into customers is expensive and inefficient in many cases.
领英推荐
The ABCs of experimental design, a few case studies
To illustrate the use of experimental design in marketing, we can consider a simple case study.
A company x, has to sell a product to other companies, and before placing it on the market it has to test three different variables: price, message and promotion.
Each of the three variables can have variations:
The total number of combinations is obtained by multiplying the levels of each attribute.?
4x2x2=16, and then placing the combinations on a map.
There is no need to test every single variable in this case.
The Xs indicate the cells to be tested, and each price is matched to either variable 1 or variable 2 (in this case message 1 or message 2, promotion 1 or promotion 2).
The 8 combinations are then tested, using different media: from traditional tools, via banner ads on the website and via e-mail messages.
Sample size must be large enough for marketers to determine the attributes, based on consumer responses.
Within a few days, the experiment results arrive and positive answers are noted for each of the eight offers tested:
Through this comprehensive overview, it becomes immediately clear how some combinations are more effective than others. Specifically, price 1, with the message emphasising power and the free gift is the solution that appeals most to consumers.
However, it is not always the right combination for the company, as it must be understood whether this solution will be optimal for revenues and turnover of the entire organisation.
Experimental design in Crayola
During the 2000s, Crayola, a company of the Binney & Smith and Hallmark group, launched a portal called Crayola. com, on which online purchases could be made.
The website is targeted mainly to families and teachers and sells art materials, as well as offering ideas for creating educational lesson plans for the classroom.
Marketing department, in order to attract new users to its website and convert them into new customers, implemented experimental design.
They identified a number of initiatives that could lead to a purchase, including sending newsletters to parents and teachers.
Marketing included five variables within the e-mail, which could influence the customer response rate, based on clicks made on the Crayola website.
These variables and their levels were:
Below some results:
It can be seen that in the case of the subject, "Crayola.com Survey", a 7.5% higher response rate was achieved than in the case of the second, "Help Us Help You."
Considering all levels of each variable, it was necessary to test 72 possible versions of the e-mail (2x3x2x3x2=72).
Since the process was inefficient and expensive, Crayola's marketing department focused on a subset of 16 e-mails representing all 72 possible combinations.
They then sent 16 different types of e-mails to customers over a 2-week period and analysed the answers.
Below are the results:
The best e-mail of the 72 possible combinations had a response rate of about 34% and was three times more effective than the worst e-mail.
Conclusions
Experimental design allows marketers to scientifically identify which campaigns work most effectively with a particular audience, in a particular market, without being approximate.
Examples considered are very simple, but can also be extended and applied to more complex cases, accurately estimating customer reactions to all possible combinations.
Applying experimental design in marketing also has certain limitations.
First of all, this approach requires proper planning, so as to include only the variables you wish to test, to the exclusion of other possibilities.
Furthermore, in order to implement such a process, the marketing department must be able to correctly interpret the data, depending on the time, market and target audience.
In addition to a basic knowledge of statistics, it is even more important to understand customer behaviour, making assumptions on which attributes should be tested and which should not.
Experimental design should be a continuous cycle of testing and learning, in order to communicate more effectively with customers, significantly increasing chances of achieving marketing goals.