A New Type Iterative Ridge Estimator: Applications and Performance Evaluations

Abstract

The usage of the ridge estimators is very common in presence of multicollinearity in multiple linear regression models. The ridge estimators are used as an alternative to ordinary least squares in case of multicollinearity as they have lower mean square error. Choosing the optimal value of the biasing parameter k is vital in ridge regression in terms of bias-variance trade off. Since the theoretical comparisons among the ridge estimators are not possible, it is general practice to carry out a Monte Carlo study to compare them. When the Monte Carlo designs on the existing ridge estimators are examined, it is seen that the performances of the ridge estimators are only considered for the same level of relationship between the independent variables. However, it is more likely to encounter different levels of relationships between the independent variables in real data sets. In this study, a new type iterative ridge estimator is proposed based on a modified form of the estimated mean square error function. Furthermore, a novel search algorithm is provided to achieve the estimations. The performance of the proposed estimator is compared with that of the ordinary least squares estimator and existing 18 ridge estimators through an extensive Monte Carlo design. In the design of the Monte Carlo, both data generation techniques were taken into account, based on the constant and varying correlation levels between the independent variables. Two illustrative real data examples are presented. The proposed estimator outperforms the existing estimators in the sense of the mean squared error for both data generating types. Moreover, it is also superior with respect to the k-fold cross-validation method in the real data examples.