Fast Penalized Regression and Cross Validation for Tall Data with the oem Package

Abstract

A large body of research has focused on theory and computation for variable selection techniques for high dimensional data. There has been substantially less work in the big "tall" data paradigm, where the number of variables may be large, but the number of observations is much larger. The orthogonalizing expectation maximization (OEM) algorithm is one approach for computation of penalized models which excels in the big tall data regime. The oem package is an efficient implementation of the OEM algorithm which provides a multitude of computation routines with a focus on big tall data, such as a function for out-of-memory computation, for large-scale parallel computation of penalized regression models. Furthermore, in this paper we propose a specialized implementation of the OEM algorithm for cross validation, dramatically reducing the computing time for cross validation over a naive implementation.

Keywords

• optimization
• expectation maximization