Hyperparameters Decoded

Article on Medium

Let's try to understand some key hyperparameters in deep learning models using an analogy from cricket. I hope it would help understand the concepts better.

1. Num_examples (Number of Examples):

In cricket, think of "num_examples" as the total number of cricket matches played by a team in a season. Each match represents a unique example. So, if a team plays 10 matches in a season, then "num_examples" is 10.

2. Epoch:

An "epoch" in deep learning is like a full cricket season for a team. During an epoch, the team (your deep learning model) faces each opponent (training example) once. Just like in cricket, where a team plays against different opponents in a season, your deep learning model processes through all the training examples to learn from them. If your team plays 10 matches (num_examples) in a season, one epoch means playing all 10 matches once.

3. Batch:

Now, let's introduce the concept of a "batch." In cricket, a team may not play all its matches back to back due to fatigue. Instead, they break the season into several "batches" of matches, playing a few matches in a week or month and a few on another. In deep learning, a "batch" is a group of training examples that your model processes together before updating its parameters. Just like cricket teams play in batches, deep learning models learn from batches of data to prevent memory issues and speed up training.

4. Batch Size:

The "batch size" is the number of matches (examples) your team (model) plays in each batch during an epoch (season). For example, if you have 10 matches (num_examples) in a season (epoch) and you decide to play 2 matches (batch size) in each batch, you would have a total of 5 batches in one epoch. Your team plays 2 matches, takes a break, plays 2 more matches, and so on until they have played all 10 matches.

In summary:

Num_examples: The total number of cricket matches (examples) in a season.

Epoch: One full cricket season, where your team plays all matches (examples) once.

Batch: A smaller group of matches (examples) played together in one go during an epoch.

Batch Size: The number of matches (examples) played in each batch during an epoch.

So, when you're training a deep learning model, you can adjust these hyperparameters to optimize the learning process, just like a cricket team might adjust their match schedule to perform better in a season.