New Release: How To Analyze The Performance of Regression Models in Production?

We’ve just released one more report. You can now use Evidently open-source library to analyse the performance of production ML models and explore their weak spots. Regression models are covered first.

What is it?

The Data Drift and Target Drift reports we released earlier explore the change of the model features and target. Both come handy when you do not have an immediate feedback loop.

Still, sooner or later, we know the ground truth or actual labels. We can then analyse the model performance directly and compare it to what we have seen in training.

For that, you can use our new Model Performance report. Currently, it is available for regression models.

The report helps you answer the following questions:

• How well does my model perform in production?
• Which errors does the model make?
• Are there specific data segments where the model performs differently?
• Did anything change compared to training?

Where does the model fail?

On top of a performance summary, the report gives a more in-depth look into the model errors. The goal is to help you identify data slices that require attention.

We look into the cases where model over- or under-estimates the target function.

Then, we calculate and visualise the model error for the different feature values inside the range. The goal is to identify if there is a relationship between the error and values of specific features.

For example, we might discover that our model consistently overestimates the target for the users coming from a given region, or that the error is very high at particular weather conditions.

How you use it?

We have a more detailed walk-though in our release blog post. Check it out to learn when and how to use the tool:

https://evidentlyai.com/blog/evidently-016-regression-model-performance

Or, pip install evidently to give it a try:

https://github.com/evidentlyai/evidently

Let me know your thoughts - working on the Classification report next!