Comparison of Least Squares, Ridge Regression and Principal Component Approaches in the Presence of Multicollinearity in Regression Analysis

Abstract

The aim of this study was to compare estimation methods: least squares method (LS), ridge regression (RR), Principal component regression (PCR) to estimate the parameters of multiple regression model in situations when the underlying assumptions of least squares estimation are untenable because of multicollinearity. For this aim, the effect of some body measurements on body weights (height at withers and rumps, body length, chest width, chest girth and chest depth, front, middle and hind rump width) obtained from totally 85 Karayaka lambs at weaning period raised at Research Farm of Ondokuz Mayis University was examined. Mean square error, R2 value and significance of parameters were used to evaluate estimator performance. The multicollinearity, between front and middle rump width which were used to estimate live weight, was eliminated by using RR and PCR. Although research findings showed that RR method had the smallest MSE and the highest R2 value, the estimates of PCR were determined to be more consistent when the importance tests of parameters were taken into account. The results showed that principal component regression approach should be used to estimate the live weight of Karayaka lambs at weaning period.

Keywords

• least squares
• ridge regression
• principal component regression
• multicollinearity
• body measurements