Linear Regression - An overview

Hi connections. In this article we will be discussing about "linear regression" model algorithm. Let us get started. Happy learning.

Linear regression by far has been one of the widely used model that has its deep root in defining other complex algorithms like neural networks. Do not know what is a "neural network"? Not to worry we will tackle that in another post. Let us jump right in.

As the name suggest, linear regression is a "supervised machine learning" algorithm to solve any regression task. By convention the name supervised machine learning refers to the task at hand which has classified labels in it. Broadly speaking, there are two variants of supervised machine learning such as "regression" and "classification".

Actually, regression refers to relation between mean of one variable with the other(s), especially in this case the output/response is "continuous" in nature. The term linear is coming into play because a "straight line" will be used to fit the data and find the relation which maps one or more independent variable with the response/target variable approximately. On a similar note, classification also falls in supervised machine learning but with an intent to classify the response/target varaible which is "discrete" in nature. We will discuss more on this in another post.

In simple terms linear regression is used to predict response/target variable based on other(s) independent variable. The model tries to find most optimal "linear function" as part of training process using labelled data(X<s>-y), which can be used to predict y on new data points(X<s>). If data contains one independent variable along with the response variable, then it is called "simple linear regression". In case of more than one independent variable the same is often considered to be "multiple linear regression". Keep in mind that our mapping model g(X) is an approximation of the actual function f(X) as there will be some errors associated with the defined model.

The objective of linear regression model is to find that mapping function that correctly maps X<s> to y. In other words, given a set of new X<s> model will be able to predict what is the value of y. This happens because the model already learnt true representation of X's to y during the training process. Statistically, the equation of linear regression is mentioned as below,

Y = Beta0 + Beta1 * X1 + error,
where,
Y = response/target (continuous)
Beta0 -> bias
Beta1 -> weight/coefficient
error -> irreducible error

NOTE:- The above equation is a simple linear regression model. In order to make it a mutiple linear regression model, we just need to extend the equation as mentioned below.

Y = Beta0 + Beta1 * X1 + Beta2 * X2 + ..... + BetaN * XN + error        

In this equation above, there are few unknowns to be known using the model. Is not it ? The beta terms are unknown to us which is why model will try to find best possible values of beta terms to predict Y. Having said this, hypotheis function for linear regression is parametrized by beta values given any value X<s>. The task of the model is to get the best fit line that predicts value of y using beta terms.

Now, that he model to set predict there will be error for each prediction as the predicted value will not be equal to actual value. Mathematically, we can write it like this,

y_actual = 100 (present in labelled dataset)
y_pred = beta0 + beta1 * x1 + error = 112
loss/error = y_pred - y_actual

NOTE:- this entails that loss needs to optimized so that predictions and actual values are nearly the same.         

Let us pause over here and digest all that we have discussed. We will continue next in another post on "how to get the line of best fit". Stay tuned.

Thanks for reading this article. I hope you learnt something new today. Happy learning.