Fitting Prediction Rule Ensembles with R Package pre

Abstract

Prediction rule ensembles (PREs) are sparse collections of rules, offering highly interpretable regression and classification models. This paper shows how they can be fitted using function pre from R package pre, which derives PREs largely through the methodology of Friedman and Popescu (2008). The implementation and functionality of pre is described and illustrated through application on a dataset on the prediction of depression. Furthermore, accuracy and sparsity of pre is compared with that of single trees, random forests, lasso regression and the original RuleFit implementation of Friedman and Popescu (2008) in four benchmark datasets. Results indicate that pre derives ensembles with predictive accuracy similar to that of random forests, while using a smaller number of variables for prediction. Furthermore, pre provided better accuracy and sparsity than the original RuleFit implementation.

Keywords

• prediction rules
• ensemble learning
• decision trees
• lasso penalty
• r