Calculating Social Media Impacts by Experimentation Approach

The experimentation approach is used to overcome limitations of the regression approach, that help isolate cause and effect relationships. Specifically, by using an experiment, we can test the effects of a possible causal variable - such as the amount of money spent on a social media campaign or the type of message used to encourage engagement - on an outcome variable such as sales or click-throughs. To conduct an experiment, we manipulate the casual variable and then compare the effects of the manipulation on the outcomes variables across different groups of consumers. For example, the floral shop could vary the amount of money spent to promote the workshop events on social media across two randomly selected groups of consumers - with one group receiving "high" spend and another receiving "low" spend on promotion. Comparing the number of event sign-ups that occur in these two groups provide insights into the returns on social media expenditures. Because other factors are held constant between the two groups, we can be more confident that the increase in sign-ups is attributable to the amount of money we spend to promote the workshop events. This digital context provides the ability to run simple experiments like this one with fast feedback. Using experiments require an outcome variables that can be linked to social media spending, such as a click-through to sign-up for an event or make a purchase or an impression of a video.

On the downside, calculating social media ROI in this way often requires a sophisticated understanding of experimental design. Further, because social media results in conversations among consumers, ensuring that a manipulated variable affects only a specific group of consumers can be challenging - that is, the impact of social media can naturally "bleed" or happen across groups.

When conforming a A/B test in a non-social environment, members of the 'A' and 'B' groups are randomly selected and you can test a feature for an outcome. In 2008 presidential race, for example, the team support Barack Obama's campaign showed one of four different buttons to visitors of the website. The buttons either said, "join us now," "learn more," "sign up now," or "sign up". The team measured which button got more clicks.

When performing an A/B test in a social media environment-meaning the test involves media that are shared between people - results can be muddled. For example, a team may choose two groups - one with the live video chat feature and one without. However, by nature, live video chat is social; if the team requires that users in the group with live video chat only chat with each other, they are limiting the users to share with only a select group of people, and the experiment is not true to nature of how the feature actually would be used. They also would not be able to test for externalities.

Thanks for reading