Introduction to Bayes Theorem

We already saw in the previous article Conditional probability, if you don't know what is conditional probability, you can read it Here, in this article we will look at Bayes Theorem.

Bayes Theorem, named after Thomas Bayes describes the probability of an event, based on prior knowledge of conditions that might be related to the event [1].

Quick reminder of conditional probability and introduction to Bayes Theorem :

If we want to find the P(A | B) we are assuming here that the set B would be now the entire sample space because it's P(A| B). So the probability of P(A | B) would be P(A∩B) divided by the P(B) "the entire sample space in that condition"

If want to estimate the P(B | A) it's the same thing , except that A is now the entire sample and the formula would be P(A∩B) divided by the P(A)

Recap :

• P(A | B) = P(A∩B) / P(B)
• P(B | A) = P(A∩B) / P(A)

If we arrange the denominator in the formula we would have :

• P(A∩B) = P(A | B) * P(B) = P(B | A) * P(A) *Remember this for later*
• P(A | B) * P(B) = P(B | A) * P(A)
• P(A | B) = P(B | A) * P(A) / P(B) , This is Bayes Theorem !

Let's have a more generalized understanding of Bayes Theorem, If we take an example of a sample space which is a collection of 3 events (Aj) that are Mutually Exclusive (the events cannot occur at the same time) and Collectivly Exaustive (if you add up the probability of all these events the result constitutes the sample space).

I already write an article on that, that you can read by clicking Here

Let's have a new event B which extends over the other three events

B has something in common with all of the 3 events.

P(B) = P(A1∩B) + P(A2∩B) + P(A3∩B)

Now let's use what I quoted to be remembered

P(B) = P(B | A1) * P(A1) + P(B | A2) * P(A2) + P(B | A3) * P(A3)

If you want to generalize this equation you will write it like this P(B) = ?Σ P(B | Ai) * P(Ai)

What we saw previously is this : P(A | B) = P(B | A) * P(A) / P(B) and what we see now is P(B) = ?Σ P(B | Ai) * P(Ai)

If we put the probability of P(B) that we saw earlier in the first equation , we will have this !

Let's take examples to understand !

Example :

We will take the first example we saw in the article "Conditional Probability"

Given that they passed the exam what is the probability it is a woman ?

P(A|B) = (P(A) x P(B|A) ) / P(B)

• What is the probability of having a woman ; 100/200 and it's equal to 0.5
• What is the probability of passing the exam ; 169/200 and it's equal to 0.845
• The probability of a woman passing the exam is 92/100 and it's equal to 0.92

Let's apply the formula:

• P(A|B) = (P(0.5) x P(0.92 ) / P(0.845)
• P(A|B) = 0.54

Let's verify:

• 92/169 = 0.54 too

Another example with the 2nd formula :

Covid 19 has taken over the world and the use of Covid19 tests is still relevant to block the spread of the virus and protect our families.

You can follow the statistics of Covid 19 on the World Health Organization website: https://covid19.who.int/

If the Covid19 infection rate is 10% of the population, and thanks to the tests we have in Algeria, 95% of infected people are positive with 5% false positive.

What would be the probability that I am really infected if I test positive?

Solution :

Parameters :

• 10% infected
• 95% Test positive while infected
• 5% False positive while non infected
• 90% not infected

We will start multiplying the probability of infection (10%) by the probability of testing positive given that be infected (95%) then we divided by the sum of the probability of infection (10%) by the probability of testing positive given that be infected ( 95%) with not infected (90%) multiplied by false positive (5%)

• P(A|B) = P(A) * P(B|A) / Σ P(A) * P(B|A)
• P(A|B) = 0.1 * 0.95 /(0.95 * 0.1) +(0.05*0.90)
• P(A|B) = 0.095 / 0.095 + 0.045
• P(A|B) = 0.678

Source :

1 - Joyce, James (2003), "Bayes' Theorem", in Zalta, Edward N. (ed.), The Stanford Encyclopedia of Philosophy (Spring 2019 ed.), Metaphysics Research Lab, Stanford University, retrieved 2020-01-17