What is Overfitting & Underfitting in Machine Learning?

Overfitting and underfitting are the two most common problems in machine learning, both of which have an impact on model performance.

The main goal of any machine learning model is to generalize well. Generalization refers to an ML model's ability to deliver an acceptable output by altering a given set of unknown inputs. It means that after training on the dataset, it can produce reliable and accurate results. As a result, the terms underfitting and overfitting must be analyzed in relation to the model's performance and whether it is properly generalizing.

Overfitting

Overfitting occurs when our machine learning model attempts to cover all of the data points in a dataset, or more than the required data points. As a result, the model starts to cache noise and erroneous values from the dataset, lowering its efficiency and accuracy. The overfitted model has a low bias and a large variance. Overfitting becomes more likely as we provide additional training to our model. The more we train our model, the more likely it is to become overfitted.

How to avoid the Overfitting in Model?

Overfitting and underfitting both hurt the machine learning model's performance. However, because overfitting is the primary cause, there are some strategies that can be used to reduce the risk of overfitting in our mode.

- Cross-Validation

- Training with more data

- Removing features

- Early stopping the training

- Regularization

- Ensembling

Underfitting

Underfitting occurs when our machine learning model fails to grasp the underlying trend of the data. The feeding of training data might be terminated early to avoid the model from overfitting; otherwise, the model may not learn enough from the training data. As a result, it might not be able to find the best match for the data's current trend. When a model is unable to learn enough from the training data, underfitting occurs, resulting in poorer accuracy and inaccurate predictions. Underfitted models have a large bias and a low variance.

How to avoid underfitting:

- By increasing the model's training time.

- By expanding the number of features available.