Supervised Machine Learning With Python: Classification. Logistic Regression

The logistic regression model is essentially one of the supervised classification algorithm family members. By calculating the probabilities with the aid of a logistic function, logistic regression analyzes the relationship between dependent and independent variables.

When referring to dependent and independent variables in this context, the dependent variable is the target class variable we intend to predict. In contrast, the independent variables are the attributes we intend to employ to do so.

Estimating probabilities in logistic regression refers to determining the likelihood that an event will occur. The shop owner, for instance, would like to foretell if the customer who entered the store will purchase the play station (as an example) or not. The shopkeeper would look at various consumer characteristics, such as gender, age, and others, to forecast the possibility that the customer would purchase a play station or not. The sigmoid curve is used to generate the logistic function with different parameters.

Since this is the first example of Logistic Regression that we are introduced to, for this example we will not use an existing dataset but will instead, define and create our own small sample dataset.

We will begin our script by importing all necessary packages into our programming script instance:

import numpy as np
from sklearn import linear_model
import matplotlib.pyplot as plt        

Next, we will define some sample data to use for the classification problem:

X = np.array([[2.0, 4.8],
              [2.9, 4.7],
              [2.5, 5.0],
              [3.2, 5.5],
              [6.0, 5.0],
              [7.6, 4.0],
              [3.2, 0.9],
              [2.9, 1.9],
              [2.4, 3.5],
              [0.5, 3.4],
              [1.0, 4.0],
              [0.9, 5.9]])

y = np.array([0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 3, 3])        

We will thereafter instantiate an object of the LogisticRegression class:

algorithm = linear_model.LogisticRegression(solver="liblinear",
                                            C=1.0)        

Next, we will proceed to fit the algorithm to the training data to create a predictive model:

model = algorithm.fit(X, y)        

Now, if we would like to visualize the output of our Logistic Regression model, we may do so as follows:

min_x, max_x = X[:, 0].min() - 1.0, X[:, 0].max() + 1.0
min_y, max_y = X[:, 1].min() - 1.0, X[:, 1].max() + 1.0

mesh_step_size = 0.02

x_vals, y_vals = np.meshgrid(np.arange(min_x, max_x, mesh_step_size),
                             np.arange(min_y, max_y, mesh_step_size))

output = model.predict(np.c_[x_vals.ravel(), y_vals.ravel()])
output = output.reshape(x_vals.shape)

plt.figure()
plt.pcolormesh(x_vals, y_vals, output, cmap=plt.cm.gray)
plt.scatter(X[:, 0], X[:, 1], c=y, s=75, edgecolors='black', linewidth=1,
            cmap=plt.cm.Paired)

plt.xlim(x_vals.min(), x_vals.max())
plt.ylim(y_vals.min(), y_vals.max())

plt.xticks((np.arange(int(X[:, 0].min() - 1), int(X[:, 0].max() + 1),
                      1.0)))
plt.yticks((np.arange(int(X[:, 1].min() - 1), int(X[:, 1].max() + 1),
                      1.0)))

plt.show()        

The output of our Logistic Regression visualization will show as follows:

The image in the plots pane displays as follows:

In this article, you do not need to be concerned with the visualization of the Logistic Regression model. I have included that for information purposes only. For now, the most important aspect is to ensure that you understand the logic behind each of these classification algorithms.

It is important to understand the methodology behind each of these algorithms and how they operate on a functional-decomposition level.