Robust Ridge Regression Approach for Combined Multicollinearity- Outlier Problem

Abstract

Ordinary least squares (OLS) offers good parameter estimates in regression if all assumptions are met. However, if the assumptions are not adhered to due to the presence of combined multicollinearity and outliers, parameter estimates may be severely distorted. Hence, robust parameter estimates were injected into the ridge regression method to produce robust ridge regression models. Therefore, the aim of this study is to investigate the performance of selected robust ridge estimators which include S, M, MM and Least Trimmed Squares (LTS) estimators via a simulation study. Laplace and Cauchy error distributions were introduced as outliers in the simulated data of various sample sizes and levels of multicollinearity. The performance of the estimation methods is investigated using criteria bias and root mean square error (RMSE). The finding indicates that Ridge LTS is the best robust ridge estimator in handling data containing both multicollinearity and outliers due to its smallest value in the RMSE. Applications of the estimators to two benchmark real-life datasets provide similar results.