Convergence

In this article, we are reviewing the three distinct and popular ways of convergence in Probability Theory.

We let X1, X2, ... and X all defined on the same probability space (Ω, T, P).

1. Simple convergence

The sequence X1, X2, ... simply converges if

This notion is too strong in most cases. For example, assume we toss a fair coin infinitely. On average we'll see a head (or tail) half of the time. Assume Xn=1 if the output is a head, and 0 otherwise, at the toss n, for any n. Then there exist an eventuality ω1="HHHHHHHH..." for which the average (X1+X2+...)/n evaluated on ω1 is 1, and an eventuality ω2="FFFFFFFF..." for which the average evaluated on ω2 is 0. This convergence does not coincide with the intuition issued from our empirical observation.

2. Convergence almost surely

The sequence X1, X2, ... almost surely (a.s.) converges if

Perhaps you may have an incredible lack of luck, you may observe indeed that the average proportion of heads (or tails) is half.

It is interesting to keep in mind with some practical criterii for the convergence almost surely.

• if the sequence ΣP(|Xn-X|>ε) converges for all ε>0, then the sequence X1, X2, ... a.s. converges.

Hint: use the Borel-Cantelli lemma.

• if ΣE|Xn-X| converges, then the sequence X1, X2, ... a.s. converges.

Hint: use the Markov inequality and the previous point.

When you perform an reductio ad absurdum, you can prove that the sequence X1, X2, ... a.s. converges if and only if

Indeed, you can start from:

(the little "c" means "complement").

3. Convergence in probability

The sequence X1, X2, ... converges in probability if

This means that the set of Xn's which move away from X is reducing to 0 when n tends to infinity.

In the example above, all the "averages" getting away from half tend to collapse in proportion.

There is a non-straightforward characterization of convergence in probability with the mathematical expectation.

• Consider a function ψ from R to R+ and which is positive, continuous, bounded, non-decreasing and (strictly) increasing in the neighborhood of 0, such that ψ(0)=0. Then the sequence X1, X2, ... converges in probability if and only if

• From this, setting ψ(x)=min(x,1), we deduce: the sequence X1, X2, ... converges in probability if and only if

This last property is often used to prove convergence in probability.

4. Comparison

To remember:

• Convergence almost surely implies convergence in probability (use the definitions to prove this)
• Conversely, if there is convergence in probability, one can extract a sub-sequence which converges almost surely.

Note: there also is the convergence in law, but it might be for next time!