Machine Learning enables computers to make predictions based on historical data. One of the most important techniques in supervised learning is Regression, which is used to predict continuous numerical values such as house prices, salaries, sales, or temperatures.
Regression forms the foundation of many real-world AI applications because it helps us understand the relationship between input variables and expected outcomes. In this blog, we’ll explore the basics of regression, essential statistical concepts, evaluation metrics, and the differences between Simple and Multiple Linear Regression. These concepts are covered in the Day 4 AI learning material.
What is Regression?
Regression is a supervised machine learning technique used to predict continuous numerical values.
Instead of predicting categories like “Spam” or “Not Spam,” regression predicts values such as:
- House Price
- Employee Salary
- Temperature
- Sales Revenue
- Stock Prices
According to the presentation, regression aims to find the relationship between the dependent variable (Y) and one or more independent variables (X) by learning a function:
Y = f(X)
Understanding the Basics Before Learning Regression
Before building regression models, it’s important to understand several fundamental concepts.
Independent Variable (Feature)
An Independent Variable (also called a feature) is the input used to predict an output.
Examples include:
- Years of Experience
- House Size
- Age
- Advertising Budget
The presentation defines the independent variable as the input feature used to predict an output.
Dependent Variable (Target)
The Dependent Variable is the value that the model attempts to predict.
Examples:
- Salary
- House Price
- Sales
- Temperature
The presentation describes the dependent variable as the output we want to predict.
Population vs Sample
Understanding data is essential before training any Machine Learning model.
Population
The complete dataset.
Example:
All employees in a company.
Sample
A subset selected from the population.
Example:
100 employees chosen for analysis.
The presentation illustrates this concept using employees in a company as the population and a selected group as the sample.
Correlation
Correlation measures how strongly two variables are related.
Three common types are:
Positive Correlation
As one variable increases, the other also increases.
Example:
More years of experience → Higher salary
Negative Correlation
As one variable increases, the other decreases.
Example:
Higher product price → Lower demand
No Correlation
There is no meaningful relationship between the variables.
The presentation introduces these three correlation types as part of the regression foundation.
Bias and Variance
Two important concepts that influence model performance are Bias and Variance.
Bias
Bias occurs when the model is too simple and cannot capture the true pattern in the data.
This often leads to underfitting.
Variance
Variance occurs when the model learns the training data too well, including noise.
This often leads to overfitting.
The presentation explains bias as error caused by an overly simple model and variance as error caused by learning too much from the training data.
Regression Evaluation Metrics
After training a regression model, we need to evaluate its performance.
The presentation introduces four widely used evaluation metrics.
1. Mean Absolute Error (MAE)
MAE calculates the average absolute difference between actual and predicted values.
Formula:
MAE = Σ |Actual − Predicted| / n
Characteristics:
- Easy to understand
- Treats every error equally
2. Mean Squared Error (MSE)
MSE squares each prediction error before averaging.
Formula:
MSE = Σ (Actual − Predicted)² / n
Characteristics:
- Penalizes large errors more heavily
- Commonly used during model optimization
3. Root Mean Squared Error (RMSE)
RMSE is simply the square root of MSE.
Formula:
RMSE = √MSE
Benefits:
- Easier to interpret because it uses the same unit as the target variable.
4. R-Squared Score (R²)
R-Squared measures how well the regression model explains the variation in the target variable.
The presentation notes that the score ranges from 0 to 1, where values closer to 1 indicate that the model explains more of the target variable’s variation.
Simple Linear Regression
Simple Linear Regression predicts a continuous value using one independent variable.
The presentation gives the equation:
Y = mX + c
Where:
- Y = Dependent Variable
- X = Independent Variable
- m = Slope
- c = Intercept
Applications of Simple Linear Regression
Simple Linear Regression is widely used in many industries.
Some common applications include:
- House Price Prediction
- Sales Forecasting
- Temperature Prediction
- Revenue Prediction
- Market Analysis
These example applications are listed in the presentation.
Limitations of Simple Linear Regression
Although simple and effective, this algorithm has several limitations.
According to the presentation, it:
- Assumes a linear relationship
- Is sensitive to outliers
- Is not suitable for classification tasks
- Performs poorly on complex patterns
Multiple Linear Regression
Many real-world problems depend on multiple input variables.
For example, predicting employee salary may depend on:
- Years of Experience
- Education Level
- Skill Set
- Job Role
Instead of using one feature, Multiple Linear Regression (MLR) predicts a continuous value using multiple independent variables.
Multiple Linear Regression Equation
The presentation introduces the following equation:
Y = b₀ + b₁X₁ + b₂X₂ + b₃X₃
Where:
- Y = Predicted Output
- b₀ = Intercept
- b₁, b₂, b₃ = Coefficients
- X₁, X₂, X₃ = Input Features
Example of Multiple Linear Regression
The presentation concludes with an example of Employee Salary Prediction, where salary is estimated using multiple input features instead of a single variable.
Stay Tuned For The Next Blog.
