What are the most common fallacies and biases that can impact predictive analytics models?

1 Confirmation bias

Confirmation bias is the tendency to seek and favor information that confirms one's preexisting beliefs or hypotheses, while ignoring or rejecting evidence that contradicts them. This can affect predictive analytics in various ways, such as selecting data that supports a desired outcome, choosing algorithms that favor certain variables, and interpreting results in a way that confirms one's expectations. To reduce confirmation bias, you should use objective and transparent criteria to select and preprocess the data, employ multiple and diverse algorithms to test and validate the predictions, and seek feedback from different perspectives. Additionally, it’s important to document any assumptions or limitations, compare the performance and robustness of different models, and acknowledge and address any gaps or flaws in the results.

2 Simpson's paradox

Simpson's paradox is a phenomenon in which a trend or relationship that appears in different groups of data disappears or reverses when the groups are combined, or vice versa. This can be caused by not taking into account confounding factors or variables that influence the outcome. Simpson's paradox can mislead predictive analytics by hiding or distorting the true effect or correlation of a variable on the outcome when the data is aggregated across groups, or by creating or exaggerating a spurious effect when the data is disaggregated into smaller subgroups. To avoid or detect Simpson's paradox, you should analyze the data at the appropriate level of granularity and account for any confounding factors that may affect the outcome. Additionally, visualizations and statistical tests can be used to compare the data across different groups and identify any inconsistencies or anomalies.

3 Overfitting and underfitting

Overfitting and underfitting are two common problems that affect the generalization and accuracy of predictive analytics models. Overfitting happens when a model is too complex or flexible, learning the noise or randomness in the data, instead of the underlying pattern or trend. On the other hand, underfitting occurs when a model is too simple or rigid, and fails to capture the complexity or variability in the data. As a result, predictive analytics can be compromised by reduced performance or reliability on new data, as well as increased risk and uncertainty of predictions. To avoid or correct overfitting and underfitting, you should use cross-validation and regularization techniques to evaluate and optimize the model, as well as feature selection and engineering methods to improve the quality and relevance of the data. This will enhance the model's ability to learn and generalize.

4 Cognitive biases

Cognitive biases are systematic errors or deviations in human judgment or reasoning, that result from heuristics, emotions, motivations, or social influences. These biases can affect predictive analytics in various stages such as framing the problem, collecting data, building the model, and interpreting the results. Some of the common cognitive biases include anchoring bias, availability bias, confirmation bias, hindsight bias, optimism bias and recency bias. To avoid or minimize cognitive biases, one should use a structured and systematic approach to define and solve the problem; use a diverse and representative sample of data; use a robust and transparent methodology to build and evaluate the model; and use a critical and objective perspective to interpret and communicate the results.