Differentiating Causal from Correlational Analysis In Our Analytic Endeavors

You’ve all heard the (worn out) phrase “correlation does not imply causation” or some variation of it. And it is quite right - just because two data points are related to one another (positively or negatively) does not mean that one “causes” the other. There are a multitude of reasons for why we should use caution when making causal inferences regarding the significant relationship between two data points. However, we should also be mindful of our liberal application of the phrase across the analysis we read. I have seen quite a few instances where analysis is assumed to be “flawed” because the authors did not establish causation in describing their results, even though the method (and purpose) of the analysis was predictive rather than causal in nature. The Harvard Business Review wrote an article addressing this confusion, stating that “the mistake leaders make here is failing to understand the distinction between prediction and causation…leaders who overlook useful patterns because they are not causal. The truth is, there are also times when a correlation is not only sufficient, but is exactly what is needed.”

So what is the difference between these two approaches? Generally speaking, to determine if causality is being examined, one element to look out for is a manipulation of variables to test cause-and-effect relationships. Usually this will be in the form of one or multiple variable(s) that are manipulated and an outcome (dependent) variable that is observed to change in response to the manipulation.? In correlational research on the other hand, variables are only observed with no manipulation or intervention implemented by researchers. This limited control means other variables may play a role in the relationship, hence the saying “correlation does not imply causation.” Experimental designs are now commonplace in private sector settings (especially in advertising and tech) and some companies would not even think of releasing a product feature without first testing its behavioral impact on a sample of their customer set. In most cases we are oblivious to these tests, not knowing if we are part of the control or experimental group in a company’s product rollout. For more insights on this trend I would recommend “The Power of Experiments,” by Michael Luca and Max Bazerman, who provide a nice detailing of experimentation in the private sector.

But where an experimental design’s value lies in its ability to shed light on the causal mechanism(s) behind a topic of interest, correlational research gives us predictive insight. That is because when two variables of interest are highly correlated, we have an opportunity to predict the outcome of one from knowing the value of the other. This is a valuable exercise in its own right as prediction is a key component of risk intelligence analysis, with practitioners providing customers with advance warning on various security issues to anticipate what is around the corner. It is also very important to the intelligence practitioner because it is always the case where the analyst is asked to make analytical inferences with missing information.?

For example, counterterrorism analysts may recognize a pattern whereby a sudden increase in online chatter from a terrorist group or its sympathizers signals the planning or execution of an upcoming attack. Political analysts may also find a relationship between the intensity of political rhetoric (divisive or inflammatory speech) and instances of politically motivated violence, such as hate crimes or civil unrest. Studies have also shown a positive relationship between large-scale migration patterns and potential security threats. I would venture to say that this was one indicator being tracked before the Russian invasion of Ukraine. The point being is that in none of these examples do we (or should we) care about causality. If the counterterrorism analyst finds a strong relationship between chatter and planning/execution of an attack, then that is a very useful piece of information to assist decision makers in the field, irrespective of what “caused” the chatter to begin with.?

But let’s say that the counterterrorism analyst wanted to find out how effective a counterterrorism messaging campaign would land on the ears of would be sympathizers? Could a particular message framing provide more engagement than others? Here the analyst could engage in experimentation and devise two different messages and measure online reaction. Or maybe the analyst assumes that the messenger delivering the message is crucial to message acceptance, and she decides to work with two types of messengers to deliver the same message and measure that reaction. These are examples of manipulation of variables that gets the analyst closer to understanding causal mechanisms. But that is only driven by the type of question she is interested in answering. Hence, what we should be clarifying is what are we trying to accomplish with the analysis we are pursuing? Is your goal to predict an outcome or establish the cause of such an outcome? For the former, correlational analysis (and leveraging regression techniques) is the right tool to gain further insight. If you are really interested in establishing causality, then you have to pursue other empirical methods that get you closer to that conclusion.

Thus, to better leverage data, leaders and practitioners need to understand the types of issues data can help solve as well as the difference between those issues that can be solved with improved prediction and those that can be solved with a better understanding of causation. Only then will they make better assessments about the research being evaluated and the analytic approach to take when pursuing their own work.