When it comes to learning the fundamentals of machine learning, linear regression is one of the most widely used statistical approaches used for data analysis and predictive modeling. In this data-driven world, linear regression is used to determine a relationship between dependent and independent variables.
A dependent variable can be understood as an outcome or result that we wish to predict and the independent variable(s) are the factors or elements that affect the outcome.
In this blog, we will learn in simple ways the concept of linear regression, types, how this technique is used in predictive modeling, limitations and real-life applications.
Understanding Linear Regression
Linear Regression is built around the concept that a linear relationship exists between the dependent and independent variables.
Here’s an explanation of the above factors.
- z is the dependent variable.
- y is the independent variable.
- b0 is the intercept.
- b1 is the slope.
The problem statement here is to find the values of the intercept and slope such that the sum of the squared differences between the predicted values of (b0 + b1y) and the actual values z in the dataset is minimum. The method to achieve this is also commonly known as the least squares method.
How is the Least Squares Regression Method Implemented?
Let’s understand how this method is implemented with the help of an example.
In the example below, we have the row y which represents the number of hours different bakers spent in baking & z represents the number of batches they were able to bake during that time frame.
Now with the help of this dummy data, let’s understand how
Calculating Slope b1
The values of x & y with horizontal bars at the top refer to the averages of all the values of y & z respectively.
Now, the sum of these calculated values can be calculated as
Calculating Intercept b0
This way, the final formula will be as follows:
Consequently, in order to compute how many batches would a baker bake in 10 hours, we would just replace the value of y with 10 and see the result as:
Types of Linear Regression
There are two types of regression calculation strategies:
Simple Linear Regression
This involves a single dependent variable and a single independent variable, just like the example of least square regression showed above.
Multiple Linear Regression
This type of regression involves multiple independent variables that come into play to determine the value of one dependent variable.
Linear Regression Challenges
Although linear regression is a great tool for predictive modeling, it does have some limitations which can hamper the accurateness and application in certain scenarios. These limitations include:
Linearity Assumption
This approach assumes that there is a linear correlation between the dependent and independent variables, which may not always hold true.
Independence Assumption
When calculations are performed, the assumption that the differences between actual values and calculated values are independent, does not always hold true.
Outliers
Outliers are values that appear far away from the data cloud (data points gathered together on a graph) and they tend to have a strong influence on the calculations.
Multicollinearity
If the independent variables presented are among themselves highly correlated, this can lead to unstable outcomes of regression coefficient and thus highly influence the final outcome.
Where Can Linear Regression be Applied?
The use of predictive modeling has been prevalent in varied fields. Some of the real-life applications of linear regression includes:
Economics
Linear regression is widely adopted by economists to analyze relationships between unemployment rates and GDP and other variable factors.
Finance
Financial experts make use of predictive modeling for stock price analysis, risk assessment and overall portfolio management.
Healthcare
Linear regression is often times implemented in medical setting to predict the outcome of certain diseases based on a patient’s overall health.
Marketing
Sales and marketing experts make use of linear regression to study customer behavior and predict the pricing and sales.
And the list goes on…
The power of linear regression can be utilized in any field as long as the understanding of its limitations and principles of function is clearly established to make informed decisions.
