Frequentist vs. Bayesian Thinking: Why Human Intelligence is Different from Artificial.

Statistics often rely on straightforward rules and shortcuts, which help people make quick decisions without delving into complex reasoning. This simplicity is why many find it more accessible compared to Bayesian thinking, which requires understanding probability updates based on new evidence.

For instance, in the Monty Hall problem, where there are three doors—one hiding a prize and the others hiding goats—switching doors after one goat is revealed increases the probability of winning. Initially, each door has a 1/3 chance. After a goat is shown, the probability of the prize being behind the remaining unopened door increases to 2/3. This counterintuitive result is difficult for many to grasp because it goes against instinctive thinking patterns. (Lopez-Martinez & Keef, 2019; Rosenhouse, 2014)

AI, on the other hand, processes data and uses Bayesian reasoning naturally. By continuously updating its beliefs based on new information, it can make decisions that align more closely with logical probabilities, avoiding human biases. This capability allows AI to achieve higher levels of accuracy and intelligence, as it learns from patterns rather than relying on cognitive shortcuts (Ghahramani, 2015).

In the realm of statistical inference and probability theory, two primary schools of thought dominate: frequentist (orthodox statistics) and Bayesian approaches. While both aim to draw conclusions from data, they differ fundamentally in their interpretation of probability and their methods of analysis (Efron, 2005).

Frequentist Thinking

Frequentist statistics interprets probability as the long-run frequency of events over many repeated trials. This approach focuses on the sampling distribution of data and aims to make inferences about population parameters based on sample data. A coin with a 50% chance of landing on heads or tails provides an excellent example.

Frequentist thinking shares some common ground with everyday reasoning. On the surface, some aspects of frequentist statistics align with how people naturally approach understanding the world around them. Like common sense, frequentist methods rely heavily on observable evidence, emphasizing the importance of what can be directly measured or seen. This resonates with the everyday notion that seeing is believing, providing a foundation that feels intuitive to many.

The idea that we can learn more by observing something multiple times is a cornerstone of frequentist thinking, and it mirrors how people often approach gaining knowledge in their daily lives. Additionally, frequentist methods tend to be skeptical of extreme or highly unlikely claims, which aligns with the common sense approach of questioning results that seem too good to be true.

Bayesian Thinking

Bayesian statistics interprets probability as a degree of belief. It uses Bayes' theorem to update prior beliefs with new evidence, resulting in posterior probabilities.

Example of Bayesian Thinking:

Your medical doctor informs you that you tested positive for a rare disease. You were not feeling well for a while. The disease affects 0.1% of the population and the doctor adds that the test has 99% accuracy. Most people might intuitively think that a positive test result means there's a 99% chance of having the disease. However, this is incorrect due to the base rate fallacy.

Here's why the actual likelihood is around 9%:

1. Prior probability: The disease affects 0.1% of the population, so the prior probability of having the disease is 0.001.

2. Test accuracy: The test is 99% accurate, which means:

- True positive rate (sensitivity): 99% of people with the disease test positive

- False positive rate: 1% of people without the disease test positive

3. Applying Bayes' theorem:

P(Disease | Positive Test) = P(Positive Test | Disease) * P(Disease) / P(Positive Test)

P(Positive Test) = P(Positive Test | Disease) P(Disease) + P(Positive Test | No Disease) P(No Disease)

= 0.99 0.001 + 0.01 0.999

= 0.00099 + 0.00999

= 0.01098

P(Disease | Positive Test) = (0.99 * 0.001) / 0.01098 ≈ 0.0902 or about 9%

This counterintuitive result occurs because the false positive rate (1%) applied to the large population of people without the disease (99.9%) outweighs the true positive rate (99%) applied to the small population with the disease (0.1%).

If a second independent lab confirms the test, the likelihood of having the disease increases significantly:

1. The probability of having the disease after the first test becomes the new prior probability.

2. We apply Bayes' theorem again with this new prior and the second test result.

P(Disease | Two Positive Tests) ≈ 0.9016 or about 90%

This dramatic increase in probability demonstrates the power of multiple independent tests in confirming a diagnosis, especially for rare diseases. Veritasium uses similar example in this video.

AI Thinking Based on Bayesian Logic: A Superior Approach

Artificial Intelligence systems, particularly those employing machine learning techniques, often leverage Bayesian principles in their decision-making processes. This approach offers several advantages over typical human reasoning, making AI thinking potentially superior in many scenarios (Ghahramani, 2015).

1. Continuous Learning and Adaptation

AI systems based on Bayesian logic excel at continuous learning. They can update their beliefs (or model parameters) in real-time as new data becomes available. This mirrors the Bayesian approach of updating prior probabilities with new evidence to form posterior probabilities (Westover et al., 2023).

Humans, on the other hand, often struggle with this continuous updating process. We tend to form opinions based on initial information and find it challenging to adjust these beliefs proportionally when presented with new evidence. This cognitive bias, known as anchoring, can lead to suboptimal decision-making.

2. Handling Uncertainty

Bayesian AI systems are inherently designed to handle uncertainty. They work with probability distributions rather than point estimates, allowing them to express degrees of certainty about different outcomes.

Humans often struggle with probabilistic thinking and tend to seek certainty even when it's not possible. We're prone to overconfidence in our predictions and judgments, especially in complex scenarios with multiple variables.

3. Incorporating Prior Knowledge

Bayesian AI can efficiently incorporate prior knowledge into its decision-making process. This prior knowledge can come from expert opinions, historical data, or previous learning experiences. The system then updates this prior knowledge with new data to form more accurate posterior beliefs.

While humans can also incorporate prior knowledge, we often do so inconsistently or with biases. We might give too much weight to recent or vivid experiences (recency bias) or struggle to integrate conflicting information coherently.

4. Avoiding Overfitting

Bayesian methods in AI naturally guard against overfitting - a common problem in machine learning where a model fits the training data too closely and fails to generalize well to new data. By considering a distribution of possible models rather than a single "best" model, Bayesian approaches can provide more robust predictions.

Humans are also susceptible to a form of overfitting, where we might draw overly specific conclusions from limited personal experiences, failing to generalize appropriately to new situations.

5. Dealing with Small Sample Sizes

Bayesian methods shine when dealing with small sample sizes or sparse data. They can still provide meaningful insights by leveraging prior information and expressing uncertainty appropriately.

Humans often struggle to reason correctly with small samples. We're prone to drawing strong conclusions from limited data, a tendency known as the law of small numbers.

6. Combining Multiple Sources of Information

Bayesian AI systems excel at combining information from multiple sources, weighing each piece of evidence according to its reliability and relevance. This allows for sophisticated multi-modal learning and decision-making.

While humans can also integrate multiple sources of information, we often do so suboptimally. We might give undue weight to certain sources based on emotional factors or struggle to reconcile conflicting information.

7. Explainable Decision-Making

Bayesian approaches in AI can provide a clear chain of reasoning for their decisions. By examining how prior probabilities are updated with new evidence, it's possible to understand why the AI system arrived at a particular conclusion (Kinnear & Wilson, 2022).

Human decision-making processes, especially in complex scenarios, are often opaque even to ourselves. We might make decisions based on intuition or gut feeling without being able to fully articulate our reasoning.

8. Long-Term Planning Under Uncertainty

Bayesian decision theory allows AI systems to make optimal decisions in complex, uncertain environments by considering the expected utility of different actions. This is particularly powerful for long-term planning scenarios.

Humans often struggle with long-term planning under uncertainty. We tend to discount future outcomes inappropriately and have difficulty accurately estimating probabilities of complex future events.

What Does This Mean?

While human intuition and creativity remain valuable, AI systems based on Bayesian logic offer significant advantages in many areas of decision-making and inference. They can process and update beliefs based on large amounts of data more efficiently and consistently than humans, handle uncertainty more effectively, and avoid many of the cognitive biases that plague human reasoning.

The human tendency to favor frequentist thinking while providing a structured approach to scientific inquiry may be the reason for the reproducibility crisis in modern science. This is a nuanced and contentious issue that strikes at the heart of current debates in scientific methodology.

Frequentist methods, with their emphasis on p-values and statistical significance, can sometimes lead researchers astray. The focus on achieving statistical significance (typically p < 0.05) can create perverse incentives. Researchers may engage in practices like p-hacking, where they manipulate data or analyses until they achieve a significant result. This can lead to the publication of false positives – findings that appear statistically significant by chance but don't represent true effects.

Moreover, the binary nature of hypothesis testing in frequentist approaches (reject or fail to reject the null hypothesis) can oversimplify complex realities. It may encourage researchers to overstate the importance of marginally significant results or to dismiss potentially meaningful effects that don't meet the arbitrary threshold of statistical significance.

The pressure to produce novel, significant results for publication can exacerbate these issues. Researchers may be tempted to run multiple analyses or collect additional data until they find a significant result, a practice known as data dredging. This increases the likelihood of false positives and contributes to the reproducibility problem.

Furthermore, the frequentist approach doesn't easily incorporate prior knowledge or account for the plausibility of hypotheses, which can lead to the pursuit and publication of implausible findings that happen to achieve statistical significance.

Humans face significant limitations when it comes to processing large amounts of experiential data in a timely and practical manner. Our cognitive abilities, while remarkable in many ways, are not well-suited for rapidly analyzing vast quantities of information or detecting subtle patterns across numerous observations. This inherent limitation of human cognition makes it challenging for us to intuitively grasp the true nature of probability and randomness, especially when dealing with large numbers or long-term frequencies.

We tend to rely on heuristics and mental shortcuts, which can lead to biases and misinterpretations of statistical information. This is precisely why formal statistical methods, including frequentist approaches, are so valuable. They provide a systematic framework for analyzing data that far exceeds what the human mind can process intuitively, allowing us to make more accurate inferences and decisions based on large-scale patterns and probabilities. In essence, statistical thinking serves as an essential tool to augment our natural cognitive abilities, enabling us to navigate complex probabilistic scenarios that would otherwise be beyond our experiential grasp.

References

Efron, B. (2005). Bayesians, Frequentists, and Scientists. Journal of the American Statistical Association, 100(469), 1-5.

Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature, 521(7553), 452-459.

Kinnear, G., & Wilson, M. (2022). Towards a model of teaching and learning with Bayesian networks and learning analytics. Journal of University Teaching & Learning Practice, 19(5), 71-94.

Lopez-Martinez, A., & Keef, P. (2019). The Monty Hall Problem. Whitman College.

Rosenhouse, J. (2014). The Monty Hall Problem: A Study. Massachusetts Institute of Technology.

Westover, M. B., Westover, K. D., & Bianchi, M. T. (2023). Medical Diagnosis Reimagined as a Process of Bayesian Reasoning and Elimination. Journal of General Internal Medicine.

Recommended Reading:

"Bernoulli's Fallacy: Statistical Illogic and the Crisis of Modern Science" by Aubrey Clayton is a highly recommended book that offers a compelling critique of frequentist statistics and advocates for a Bayesian approach. Clayton, a mathematician and philosopher of science, presents a well-researched account of the history of probability and statistics, explaining how the current statistical orthodoxy came to be and why it's problematic. The book argues that there's a fundamental flaw in the statistical methods used across experimental science, contributing to the reproducibility crisis in various disciplines.

Clayton provides an accessible explanation of complex mathematical concepts, making the book suitable for readers interested in understanding the statistical methods that shape our understanding of the world. He contends that adopting a Bayesian approach - incorporating prior knowledge when reasoning with incomplete information - is necessary to address the current crisis in scientific methodology. The book has received praise from respected figures in the field, including Andrew Gelman and Persi Diaconis, for its clarity and importance in highlighting the uses and abuses of statistics.