AI essentials - Probability crash course

Probability and our world

Probability theory is a powerful and an important tool in forecasting and decision making.

We live in a world where our tomorrow is shaped by probabilities. We are surrounded by uncertain events and our lives are affected by decisions made based on these events. Uncertainty exists in the nature. Probability of an event dictates the future. What is the probability that it will rain today in Bangalore? What is the probability of the next COVID wave hitting where and when? What is the probability of alien attack?

Probability theory and tools help us make informed decision and educated guesses. These help us plan better for the future. Make contingency plans before a drought or prepare well for responding to an upcoming surge in COVID cases.

Businesses are fraught with dilemmas and uncertain events. Government regulations, market sentiments, market behavior, threat from the competition, uncertainty of benefit from opportunities and investments, etc. make decision making difficult. This can quickly turn into shooting in the dark situation without an appropriate framework. Professionals use frameworks for anticipating and managing uncertainties. In broader sense, this activity is more commonly known as risk identification and risk mitigation. In product engineering, typically, the same is known as identifying failure modes and mitigating the failure risk using control actions.

Probability based tools are used to take into account the known or the unknown uncertainties at the planning stage as well as at the decision-making stage.

Notes from crash course

Mutually Exclusive events

Two or more events that cannot occur simultaneously.

Totally Exclusive events

Out of the probable event pool (a.k.a. sample space), at least one of the events will occur.

Independent events

When occurrence of an event doesn’t depend on occurrence or non-occurrence of another event.

Probability of an event A = probability of an event A given B occurs = probability of an event A given B doesn’t occur

Dependent events

When occurrence of an event depends on occurrence or non-occurrence of another event.

Probability of an event A ≠ probability of an event A given B occurs ≠ probability of an event A given B doesn’t occur

Classical approach

Probability of an event A out of n equally likely mutually exclusive outcomes is 1/n.

E.g. probability of drawing a spade out of a deck of cards = 13/52 = ?

Probability of a head on a fair toss of a fair coin = ?

Relative frequency approach

Probability is represented as the fraction of times an event has occurred in justifiably large number of past trials.

E.g. Probability of a defect is ratio of actual defects found and total manufactured items

Subjective approach: Probability is an estimated value based on previous data and subjective judgement of experts?

Conditional probability theory

The likelihood of an occurrence of an event may depend on another event’s occurrent or non-occurrence. One event is said to be related to another event.

E.g. Agriculture yields have direct relation with the climate events. A product’s success can have a direct or indirect relation with market conditions.

Addition Theorem

For given n mutually exclusive events, probability that at least one of them will occur is equal to sum of their probabilities.

For a general case with not necessarily mutually exclusive events, the following holds true:

Probability of A OR B = Probability of A + Probability of B – Probability of A AND B

i.e. P{A U B} = P{A} + P{B} – P{A ? B}

Multiplication Theorem

Probability of A AND B = Probability of A given B * Probability of B

i.e. P{A?B} = P{A|B} * P{B} for independent {A, B} events, this simplifies to P{A} * P{B}

Practical application in automotive

This concept is widely used at improving automotive system safety. ISO26262 Functional Safety standard requires redundancies in SW and HW to meet ASIL level requirements for a given system.

How does the redundancy help? Let’s assume that a Sub-System A has failure probability of 0.01 or 1%. Or 1 in 100 will fail.?

Solution: A system with two independent (redundant) Sub-Systems A1 and A2 will now have a failure probability = 0.01 * 0.01 = 0.0001. Or 1 in 10000 will fail. The added redundancy has increased system’s reliability by power of 2.

Bayes Theorem

Gives method to incorporate historical knowledge.

Previous knowledge on an event is used as reference a.k.a. prior probability. Latest knowledge on an event is used with the prior probability to arrive at new estimate of probability a.k.a. posterior probability.

Please feel free to share your feedback and suggestions.

Best,

Sanjay

?