Role of SVM model in current data science

Why to read this?

One of the most prevailing and exciting supervised learning models with associated learning algorithms that analyse data and recognise patterns is Support Vector Machines (SVMs). It is used for solving both regression and classification problems.

Technical explanation

Support vector machine classifiers provides good generalisation capability. It try to solve this problem by fitting a line to the model that tries to maximise the distance to the closest training instances (known as Support Vectors), so that the margin parallel to the decision boundary line is as wide as possible. 

Explanation using an analogy

Think of the decision boundary like the centre of a country road and the data as trees, with trees of different types at each side of the road. What SVMs try to do is to find widest road as possible that separates our two kinds of trees, so that we can safely drive through it while feeling secure. It does this by trying to maximise the margin.

SVM parameters

SVM algorithms use a set of mathematical functions that are defined as the kernel. The function of kernel is to take data as input and transform it into the required form.

Kernel is the SVM parameter. The linear, polynomial and RBF or Gaussian kernel are examples.

How SVM can solve non-linear problems?

In below picture, two data-sets are not linearly separable. However in 3-D, it is linearly separable(separable by 3-D hyperplane). 

SVM kernel functions help to achieve above case. 

Non linear SVM

Its simply SVM which uses non-linear kernel(same as above section). Refer here for detail. Below picture shows SVM classifier with polynomial kernel.

SVM for regression problem

• It is called SVR (Support Vector Regression) and it supports both linear and non-linear regressions.
• In short, this approach allows the a small deviation(say epsilon) and considers error only if the deviation goes beyond this value 

SVM hyperparameters tuning

It is done via trial and error(exhaustive search). 

Please refer here for the Gridsearch approach

Use in machine learning

• SVM can be used for both kind of datasets - linear and non-linear
• It is useful for both types of problem - Classification and regression

For example classification using SVM, refer colab code here

Relevance with neural networks

Neural network can do what SVM can do. However, for some cases, SVM training is faster compared to neural networks.

Point to remember

For non-linear problem, polynomial function can be treated as linear function in higher dimension. However, same is not true for exponentiation function (y= e^x). Note that mathematical formula of e^x is infinite series and so, the corresponding linear equation will be infinite dimensional. There is no SVM solution for infinite dimension.

Reference

https://www.vebuso.com/2020/02/a-top-machine-learning-algorithm-explained-support-vector-machines-svms/

https://towardsdatascience.com/support-vector-machines-explained-25a685e4d228

https://youtu.be/qnEZ1rF0H1Y

https://images.app.goo.gl/6tgV3nGdjKk5sReM7

https://www.researchgate.net/post/Diffference_between_SVM_Linear_polynmial_and_RBF_kernel

https://data-flair.training/blogs/svm-kernel-functions/

https://youtu.be/qnEZ1rF0H1Y?t=3568

https://colab.research.google.com/drive/1fV31Ma0mF6SVsmRl5P9W5I8qJoNbrh8j?usp=sharing

https://medium.com/@zxr.nju/what-is-the-kernel-trick-why-is-it-important-98a98db0961d

https://images.app.goo.gl/DCNgkzCiAPkpu1zb9

https://coursera.org/share/6ca58dbfbaa56ef7cb0de0dcdf32ff38

https://coursera.org/share/eceb52dc4359f0dff8d087a54fb3f00c

https://coursera.org/share/946b701f8ded6d4f893f90320386f3a6

https://www.amazon.in/Hands-Machine-Learning-Scikit-Learn-TensorFlow/dp/1491962291

https://www.isical.ac.in/~caiml/courses/workshop002/

https://www.educba.com/support-vector-regression/

https://stats.stackexchange.com/questions/13194/support-vector-machines-and-regression

https://www.geeksforgeeks.org/svm-hyperparameter-tuning-using-gridsearchcv-ml/

https://images.app.goo.gl/fRobP5VDq7mmJdKDA