ML Algorithms equations made simple

Linear Regression:

Equation: y = β? + β?x? + β?x? + ... + β?x? + ?

Explanation: Linear regression models the relationship between dependent variable (y) and independent variables (x?, x?, ..., x?) by fitting a linear equation. The coefficients β?, β?, β?, ..., β? represent the intercept and slopes of the line, while ? denotes the error term.

Python Code:

from sklearn.linear_model import LinearRegression

# Create a Linear Regression model

model = LinearRegression()

# Fit the model to the training data

model.fit(X_train, y_train)

# Predict using the trained model

y_pred = model.predict(X_test)

Logistic Regression:

Equation: p(y=1|x) = 1 / (1 + exp(-z)), where z = β? + β?x? + β?x? + ... + β?x?

Explanation: Logistic regression models the probability of the dependent variable (y) belonging to class 1 given the independent variables (x?, x?, ..., x?). The logistic function transforms the linear combination of coefficients and input variables into a value between 0 and 1, representing the probability.

Python Code:

from sklearn.linear_model import LogisticRegression

# Create a Logistic Regression model

model = LogisticRegression()

# Fit the model to the training data

model.fit(X_train, y_train)

# Predict using the trained model

y_pred = model.predict(X_test)

Decision Trees:

Decision trees involve a tree-like model of decisions and their possible consequences. Each internal node represents a feature or attribute, each branch represents a decision rule, and each leaf node represents the outcome.

Python Code:

from sklearn.tree import DecisionTreeClassifier

# Create a Decision Tree model

model = DecisionTreeClassifier()

# Fit the model to the training data

model.fit(X_train, y_train)

# Predict using the trained model

y_pred = model.predict(X_test)

Random Forests:

Random forests are an ensemble learning method that combines multiple decision trees. Each tree is trained on a random subset of the training data, and the final prediction is determined by aggregating the predictions of individual trees.

Python Code:

from sklearn.ensemble import RandomForestClassifier

# Create a Random Forest model

model = RandomForestClassifier()

# Fit the model to the training data

model.fit(X_train, y_train)

# Predict using the trained model

y_pred = model.predict(X_test)

Support Vector Machines (SVM):

Equation: w?x + b = 0

Explanation: Support Vector Machines aim to find the hyperplane that separates the classes in the feature space with the maximum margin. The equation represents the decision boundary, where w is the weight vector, x is the feature vector, and b is the bias term.

Python Code:

from sklearn.svm import SVC

# Create a Support Vector Machine model

model = SVC()

# Fit the model to the training data

model.fit(X_train, y_train)

# Predict using the trained model

y_pred = model.predict(X_test)

K-Nearest Neighbors (KNN):

Equation: y = mode(y?, y?, ..., y?)

Explanation: KNN classifies a new data point based on the majority class of its k nearest neighbors. The equation represents the majority voting process, where y represents the predicted class of the new data point.

Python Code:

from sklearn.neighbors import KNeighborsClassifier

# Create a K-Nearest Neighbors model

model = KNeighborsClassifier()

# Fit the model to the training data

model.fit(X_train, y_train)

# Predict using the trained model

y_pred = model.predict(X_test)

Naive Bayes:

Naive Bayes classifiers are based on Bayes' theorem and assume independence among features. There are different types of Naive Bayes classifiers, such as Gaussian Naive Bayes, Multinomial Naive Bayes, and Bernoulli Naive Bayes, each suited for different data types.

Python Code (Gaussian Naive Bayes):

from sklearn.naive_bayes import GaussianNB

# Create a Gaussian Naive Bayes model

model = GaussianNB()

# Fit the model to the training data

model.fit(X_train, y_train)

# Predict using the trained model

y_pred = model.predict(X_test)

Neural Networks:

Neural networks consist of interconnected nodes (neurons) organized in layers, where each neuron applies a nonlinear activation function to its inputs. The weights and biases of the neurons are learned during training.

Python Code (using Keras):

from tensorflow import keras

from tensorflow.keras import layers

# Create a Neural Network model

model = keras.Sequential()

model.add(layers.Dense(64, activation='relu', input_shape=(input_dim,)))

model.add(layers.Dense(64, activation='relu'))

model.add(layers.Dense(num_classes, activation='softmax'))

# Compile the model

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# Fit the model to the training data

model.fit(X_train, y_train, epochs=10, batch_size=32)

# Predict using the trained model

y_pred = model.predict(X_test)