Darting into ML: A Beginner's Guide to Loss Functions

Article on Medium

What is a loss function in Machine Learning?

Imagine you're playing darts:

Target: Your target is to hit the bullseye right in the center.

Your Throws: Each throw represents a prediction made by your model.

Distance from Bullseye: The distance between where your dart lands and the bullseye is like the "error" or "loss" of your prediction.

Perfect Throw: Ideally, you want to make throws (predictions) where your darts (predictions) are right on target (actual values).

Now, in the language of machine learning:

Think of a "loss function" as a measure of how far off your predictions are from the actual values or outcomes. It's like a score that tells you how well or poorly your model is performing. The goal in machine learning is to minimize this score, meaning you want your predictions to be as close as possible to the actual values.

Loss Function: This is like a ruler that measures how far off each throw (prediction) is from the bullseye (actual value).

Minimizing Loss: Your goal is to find the best technique or strategy (model) that minimizes the overall distance of all your throws (predictions) from the bullseye (actual values).

Different machine learning problems and models have different loss functions. Here are a couple of common ones:

Mean Squared Error (MSE):

This is like averaging the squared distances from the bullseye. Squaring helps to give more weight to larger errors.

Binary Cross-Entropy (Log Loss):

This is used for problems where you have two classes (binary classification). It measures the "surprise" of your predictions compared to the actual outcomes. It's like measuring how surprised you are each time you hit or miss the bullseye. A very unexpected miss might hurt your score more.

Mean Absolute Error (MAE):

MAE measures the average absolute difference between your dart throws and the bullseye. It is helpful when you want a clear understanding of the average magnitude of errors without the squaring effect. It treats all errors equally, making it robust to outliers in your predictions.

In the context of machine learning, the term "loss function" is sometimes also referred to as 'cost function', 'error function' or 'risk function'.