Decoding AI: Markov Chain

What is Markov Chain? A Markov Chain is a discrete stochastic process (such as a random walk) in which the probabilities of occurrence of various future states depend only on the present state of the system or on the immediately preceding state and not on the path by which the present state was achieved.

What is Markov Chain Model? A statistical model that represents transitions from one state to another on a state space.

Here is a practical example for a Markov Chain:

Let’s build a simple weather model that will predict whether the day will be sunny, cloudy, or rainy. A Markov Chain can be used to represent this weather model with three states: Sunny, Cloudy, and Rainy.

The diagram shows the probabilities of transitioning from one weather state to another.

If today is Rainy day the next day would be:

-??????? Rainy with 60% probability.

-??????? Sunny with 10% probability.

-??????? Cloudy with 30% probability.

If today is Sunny day the next day would be:

-??????? Rainy with 10% probability.

-??????? Sunny with 50% probability.

-??????? Cloudy with 40% probability.

If today is Cloudy day the next day would be:

-??????? Rainy with 50% probability.

-??????? Sunny with 40% probability.

-??????? Cloudy with 10% probability.

In the early 20th century, the Russian mathematician Andrey Markov (1856 - 1922) developed the concept of Markov Chains. For many years, Markov Chains were used mainly in theoretical mathematics but later were used in combination with the Monte Carlo method for applications in nuclear physics in the 1940s.

Markov Chain Modeling is heavily used in real-world scenarios such as Google's PageRank algorithm, predicting stock prices, analyzing DNA sequences in bioinformatics, speech recognition in natural language processing, and simulating user behavior in social networks.

For Generative AI, Markov Chains are used to generate sequences of data, such as text, music, or images, buy modeling the probability of transitioning from one state to another one.

For NLP, a Markov chain can be used to generate a sequence of words that form a complete sentence, or a hidden Markov model can be used for named-entity recognition and tagging parts of speech.

For machine learning, Markov decision processes are used to represent reward in reinforcement learning.

In other words, all current AI technologies are employing the concept of Markov Chains developed more than 100 years ago.