Partial Least Squares Regression Trees for Multivariate Response Data With Multicollinear Predictors

Abstract

Some problems arise in analyzing massive complex data consisting of multivariate response variables and a large number of multicollinear predictor variables, especially when the sample sizes compared to the number of predictors are small. Rather than ordinary linear regression modeling approaches, latent variable regression modeling approaches such as partial least squares regression can be used to capture the relationship between the response and predictor variables for such cases. However, for complex nonlinear relationships between the predictor and the response variable, the performance of inference and prediction using regression modeling approaches can be deflated. Regression trees can capture such complex relationships. Thus, we develop a partial least squares tree modeling algorithm that detects complex relationships and makes precise predictions by integrating the merits of partial least squares and regression trees. It is shown that it has better predictive performance than other methods through simulation and it is demonstrated that it generates interpretable predictive models with real data of usedcar and orthognathic surgery.

Keywords

• Multicollinear
• partial least squares
• complex nonlinear relationships