Supervised Machine Learning With Python: Classification. Support Vector Machines

A support Vector Machine (SVM) is essentially a supervised machine learning technique that may be applied to both classification and regression. The primary idea behind SVM is to plot each data point as a point in n-dimensional space with each feature’s value represented by a specific coordinate. The features we would have in this case would be n. The following is a straightforward graphical illustration of the SVM concept:

There are two elements in the diagram above. Therefore, we must first plot these two variables in a two-dimensional space where each point has two coordinates, which are referred to as support vectors. The line divides the data into two distinct categories. The classifier would be this line.

Here, we’ll use the iris dataset and scikit-learn to develop an SVM classifier. The SciKit Learn package provides us with the sklearn.svm sub-package and the sklearn.svm.svc for building Machine Learning classification models.

In this Machine Learning problem, we will use the SVM classifier to determine the class of iris plants. We will be using a dataset that is built into, and available to us from, the SciKit Learn Library.

THE IRIS DATASET

For this Machine Learning task, we will make use of the iris dataset, which has three classes with 50 observations each and each class refers to a certain kind or class of iris plant. Sepal length, sepal width, petal length, and petal width are the four characteristics present in each case. We shall now utilize Support Vector Machines Algorithm to construct a Machine Learning Classification Model.

THE KERNEL

It is a method that SVM employs. These functions essentially change low-dimensional input space into a higher-dimensional space. It transforms intractable problems into solvable ones. Any of the following can be used as the kernel function: linear, polynomial, rbf (Radial Basis Function), and sigmoid. We’ll apply the linear kernel in this example.

As always, we begin our Python script by importing all necessary packages for us to use:

import pandas as pd
import numpy as np
from sklearn import svm
from sklearn import datasets
import matplotlib.pyplot as plt        

Next, we will load the dataset into our system memory:

iris_dataset = datasets.load_iris()        

Next, we will be required to plot the boundaries of the Support Vector Machine using the original data points. For this, we may make use of the first two features. We are effectively creating a mesh to plot:

X = iris_dataset.data[:, :2]
y = iris_dataset.target

x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1

h = (x_max / x_min) / 100

xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))

X_plot = np.c_[xx.ravel(), yy.ravel()]        

Next, we will set a value for the regularization parameter:

C = 1.0        

We will thereafter instantiate an SVC Object and proceed to train our model as follows, and plot the model using Python’s popular visualization package, MatPlotLib:

algorithm = svm.SVC(kernel='linear',
                    C=C,
                    decision_function_shape='ovr').fit(X, y)

Z = algorithm.predict(X_plot)

Z = Z.reshape(xx.shape)

plt.figure(figsize=(15, 5))
plt.subplot(121)
plt.contourf(xx, yy, Z, cmap=plt.cm.tab10, alpha=0.3)
plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.Set1)
plt.xlabel('Sepal Length')
plt.ylabel('Sepal Width')
plt.xlim(xx.min(), xx.max())
plt.title('Support Vector Machine Classification Model')        

The output to the above blocks of code will show as follows:

The Support Vector Machine visual that we observe on the screen in the plots pane is as follows:

And with that, we have effectively trained and visualized the logic behind a Support Vector Machine Classification Model based on SciKit Learn’s built-in Iris dataset.

I do hope that you have valuable takeaways from this post. Be sure to like, share, and follow my blog for more such knowledge.

Thank you for your time.

Shivek Maharaj