In the realm of machine learning and statistics, regression metrics are crucial for evaluating how well our models perform. When we build a regression model, our primary goal is to make accurate predictions. To assess the effectiveness of these predictions, we rely on various metrics such as Mean Squared Error (MSE), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and R2 Score. In this blog post, I will delve into each of these metrics, explaining their significance, calculation methods, and when to use them.
1. Mean Absolute Error (MAE)
Mean Absolute Error (MAE) is one of the simplest and most intuitive metrics we can use. It measures the average absolute difference between the predicted values and the actual values. The formula for MAE is:
$$\text{MAE} = \frac{1}{N} \sum_{i=1}^{N} |y_i - \hat{y}_i|$$
Where:
yi is the actual value,
y^i is the predicted value,
N is the number of observations.
MAE gives us a straightforward interpretation: it tells us how far off our predictions are on average. The unit of MAE is same as output column. It us robust to outliers too as compared to others, but it is not diffrentiable function. A lower MAE indicates better model performance. However, one limitation of MAE is that it treats all errors equally and does not penalize larger errors more than smaller ones.
2. Mean Squared Error (MSE)
Mean Squared Error (MSE) takes a different approach by squaring the errors before averaging them. This means larger errors have a disproportionately higher impact on the MSE compared to smaller errors. The formula for MSE is:
$$\text{MSE} = \frac{1}{N} \sum_{i=1}^{N} (y_i - \hat{y}_i)^2$$
The squaring of errors serves two purposes: it ensures that all error values are positive and emphasizes larger discrepancies. While MSE is sensitive to outliers and can be useful when we want to minimize large errors, it can also be misleading if our dataset contains significant outliers. However, it is differentiable and can be minimized easily.
3. Root Mean Squared Error (RMSE)
Root Mean Squared Error (RMSE) is simply the square root of MSE:
$$\text{RMSE} = \sqrt{\text{MSE}} = \sqrt{\frac{1}{N} \sum_{i=1}^{N} (y_i - \hat{y}_i)^2}$$
By taking the square root, RMSE brings the error metric back to the same unit as the original data, making it easier to interpret. Like MSE, RMSE penalizes larger errors more heavily than smaller ones, which can be advantageous in many scenarios where large errors are particularly undesirable.
4. R2 Score
R2 Score, also known as the coefficient of determination, provides a measure of how well our independent variables explain the variability of the dependent variable. It tells how well our model is performing. The formula for R2 Score is:
$$R2 Score = 1 - \frac{\text{SSR}}{\text{SSM}}$$
Where:
SSR is the sum of squared errors on regression line,
SST is the total sum of squared errors on mean line.
for R2 Score = .70 means that our input columns are able to explain 70% variance in output column.
R2 values range from 0 to 1:
An R2 of 0 indicates that our model does not explain any variability in the target variable,
An R2 of 1 indicates perfect prediction, generally not possible.
While R2 is useful for understanding model performance, it can sometimes be misleading, especially in cases where overfitting occurs or when comparing models with different numbers of predictors.
When to Use Each Metric
MAE: Use when you want a simple interpretation and are less concerned about large errors.
MSE: Opt for this when you want to heavily penalize larger errors.
RMSE: Choose RMSE when you need an interpretable metric in the same units as your target variable.
R2 Score: Use R2 Score to understand how well your model captures variability but be cautious about its limitations.
Conclusion
In conclusion, after understanding all the above-mentioned options, selecting appropriate regression metrics like MAE, MSE, RMSE, and R2 Score is essential for evaluating our models effectively. Each metric has its strengths and weaknesses depending on the specific context and objectives of our analysis. By leveraging these metrics wisely, we can enhance our model's performance and make more informed predictions in real-world applications.
As we continue exploring regression analysis and its applications in machine learning, I encourage you to experiment with these metrics using your datasets to gain deeper insights into their behavior and implications for your models.
Thanks!