Stop Using This Image in Bias-Variance Trade-off Explanation
Ivan Reznikov
PhD, Principal Data Scientist || O'Reilly Book Author || TEDx/PyCon/GITEX Speaker || University Lecturer || LangChain, Large Language Models (LLMs) and Generative AI || 30K+ followers
Recently, reading a lecture at Middlesex University Dubai, I was asked a question regarding Bias-Variance Trade-off. The students didn't quite understand, how to get from pictureA to pictureB:
Their confusion was coupled with the difficulty of understanding how 4 scenarios in pictureA turn into 3 scenarios in pictureB. Another question was about the High Bias - High Variance models. To be honest it took me some time to think of such an example (want to hear your thoughts in the comments, so no spoilering my ideas).
I believe it's time to stop using the 4-target picture. If you feel, that the aims are the best way to explain bias and variance, consider using the following Venn diagram on the right.
As you might notice, with this approach, it becomes much easier to map low variance-bias explanations with underfit-sweet-overfit plots:
I'm not a designer, so there are plenty more improvements, that can be made. But from the conceptual point of view, I think this was of illustrating the idea of bias-variance tradeoff is more straightforward.
Engineering professor focused on open source software and education
3 年The four-target image correctly describes how you can measure something e.g. temperature, force, displacement, current, voltage, etc. When you extend this concept to a model that is reducing measured data to a set of parameters. The best case scenario is that the model fits your data, but not your noise. I don't think there is a case where you can create a model that "reduces bias" in the measurements though.
Data Science and AI/ML Engineer
3 年The one with circles has been overly used, however, it could be more clear to non-technicals or for high-level of explanation.
Principal Data Scientist at Memorial Sloan Kettering Cancer Center
3 年I think the picture with four circles is pretty good and really doesn’t need to be condensed into 1 picture to make someone understand the concept. Rather, a simple numerical example with some made up data could have been better. This is based on my personal experience talking about this topic