EP 3: Random Variables | Paper 1: A Neural Probabilistic Language Model

In continuation to: Paper 1: A Neural Probabilistic Language Model

Hello Readers,

Just wanted to share a little revelation I had during my morning workout today. I've never been a fan of running, and honestly, it's not my go-to form of exercise. Instead, I keep things simple with just a humble kettlebell. I have been using this Symactive Neoprene Coated Solid Kettlebell from Amazon.

As I was going through my routine, swinging and lifting that chunk of iron, it hit me – simplicity is where the fun is at. There's something oddly satisfying about the straightforward, no-frills approach to fitness. While others might be pounding the pavement, I find joy in the simplicity of my kettlebell workout.

Sure, I can run, but it doesn't bring me the same joy and satisfaction. It's like finding your own groove in a world full of different workout trends. Sometimes, all you need is a basic tool and a solid routine to keep things enjoyable and effective.

So here's to keeping it simple, finding what works for you, and enjoying every swing of that kettlebell. After all, fitness is a personal journey, and there's no one-size-fits-all approach. Cheers to simple, satisfying workouts that make you look forward to getting up and breaking a sweat! And shouldn’t that be the case for learning about Artificial Intelligence as well.

Stay fit and keep it simple.

What is Random Variables?

A random variable is like a special kind of number that we can't predict exactly. It's a way of turning uncertain events into something we can work with in math. Imagine it as a number that can take different values based on chance or randomness.

Examples for Layman:

1. Coin Toss Outcome:Imagine you flip a coin. If we say "let X be the number of heads," then X is called a random variable. It could be 0 (if it lands as tails) or 1 (if it lands as heads).
2. Rolling a Die:Let Y be the number shown on a six-sided die. Y is a random variable because it depends on the randomness of the die roll and could be any number from 1 to 6.
3. Weather Temperature:Suppose Z is the temperature tomorrow. It's a random variable because we don't exactly know what it will be. It might be 70°F, 75°F, or any other temperature.
4. Number of Cars Passing a Junction:Let W be the number of cars that pass a junction in the next minute. W is a random variable because it changes and we can't precisely predict how many cars will pass.W=0: No cars pass the junction in the next minute.W=3: Three cars pass the junction, indicating moderate traffic.W=10: Ten cars pass, suggesting heavy traffic.W=1: Only one car passes, indicating a brief pause in traffic.W=5.5: A continuous variable representing an average, acknowledging variability over a longer period. These examples showcase different scenarios for the random variable W, representing the number of cars passing a junction in a minute.

In these examples, the random variable is a way of describing the uncertainty or variability in different situations using numbers. It helps us understand and work with unpredictable events in a mathematical way.

Random variables can be of 2 types.

1. Discrete Random Variables
2. Continuous Random Variables.

Discrete Random Variable:

Let’s understand it with an example related to NLP:

• Example: The number of times a specific word appears in a document.
• Explanation: Let X be the discrete random variable representing the count of occurrences of the word "apple" in a given text document. Each document has a different count, making it a discrete random variable.
• Math Problem: Find the probability of X for a document where "apple" appears 0, 1, 2, or 3 times.

Continuous Random Variable:

Let’s modify the above example slightly, to understand the continuous random variables.

• Example: This time we are considering the length of words in a document.
• Explanation: Let Y be the continuous random variable representing the length of a randomly chosen word. The length can take any non-negative real value, making it a continuous random variable.
• Math Problem: Find the probability density function (PDF) of Y for the length of words in a document, given that the average word length is 5.5 characters with a standard deviation of 2.

My go to book for understanding these concepts is: Probability & Statistics for Engineers & Scientists. I even solved all the exercise problems.

The post continues here: https://mathx.substack.com/p/random-variables-paper-1-a-neural

We will also create a Pytorch example to explain the difference and much more.

Thank you for the time.