One Minute Overview of Support Vector Regression (SVR)

For those who have followed?#52weeksofdatascience?since the start of the year — I am upgrading it to a weekly newsletter! In this first week, I am sharing the?enhanced content of the original six posts daily, reverting to a weekly schedule after that.

The?#52weeksofdatascience?newsletter covers everything from Linear Regression to Neural Networks and beyond. So,?if you like Data Science and Machine Learning, don’t forget to?subscribe!

Level 1 - One Minute Overview for Data & Analytics Executives and Curious Minds

Category:?Supervised Learning (i.e. requires labelled/target training data)

Sub-category:?Regression (i.e. used to predict continuous target)

Main Idea:?SVR is designed to model non-linear relationships between variables, and it provides flexibility in controlling the model’s robustness/sensitivity through hyperparameters.

A simple way to think about SVR is to imagine a “tube” with an estimated function (hyperplane) in the middle and boundaries on either side defined by ε (epsilon).?

The algorithm’s goal is to minimize the error by identifying a function that puts more of the original points inside the “tube” while at the same time reducing the “slack” (distance to the points outside the “tube”).

The below image demonstrates the SVR model’s prediction of a house price based on the distance from the nearest MRT station. As mentioned earlier, model hyperparameters (epsilon and C) can be adjusted to control the robustness/sensitivity of the model, i.e. to reduce/increase “waviness.”?

Everyday use cases: SVR can be used for similar problems as Linear Regression, where you need to predict a continuous output. It is handy when you have non-linear relationships in your data. Examples would be a house price based on its size and location, an asset price based on its characteristics, etc.

Level 2 - for Aspiring Data Scientists

I have written an?in-depth article?published on Towards Data Science explaining the inner workings of SVR. The article can be found?here.

Level 3 - for Data Science and Analytics Professionals

You can find a Jupyter Notebook with a complete Python code on my?GitHub repository, which you can use to create your own SVR models.