Bagging-based heteroscedasticity-adjusted ridge estimators in the linear regression model

Abstract

The existence of multicollinearity between independent variables and heteroscedastic error has a colossal impact on the performance of the ordinary least square (OLS) estimator and its covariance matrix. In this study, new bootstrap aggregation (i.e. bagging) based ridge estimators are introduced to circumvent multicollinearity by controlling the influence of heteroscedastic error terms. These estimators are a novel combination of bagging and heteroscedastic-adjusted ridge (HAR) estimators. The performance of bagging-based HAR estimators is evaluated using intensive Monte Carlo simulations by considering multicollinearity with low, high, and severe degrees of heteroscedasticity. The findings reveal that at severe heteroscedasticity, the performance of the proposed HAR-HK∗, HAR-LW∗, HAR-HSL∗, and HAR-KMS∗ estimators is 26.3 %, 81.5 %, 90.1 %, and 30.5 % better than their HAR-HK, HAR-LW, HAR-HSL, and HAR-KMS counterparts when the collinearity level is 0.80, sample size is 25, and the number of independent variables is 5. Similar improvements are observed when heteroscedasticity levels are low and moderate, with different specifications for sample size, collinearity level, and number of independent variables. Overall, the bagging-based HAR estimators are efficient and perform better than the baseline HAR estimators. The real-life applications are illustrated using livestock and passenger car mileage data. The outcomes show that the suggested HAR-LW∗ (PRESS = 10.5028) and HAR-KMS∗(PRESS = 0.0497) perform best on the passenger car and livestock data respectively. The improvements will be helpful in efficiently handling the estimation problem when the two challenging issues are present in the data. © 2025

Keywords

• bagging
• heteroscedasticity
• mean squared error (mse)
• multicollinearity
• ridge regression