Solving High-Dimensional Problems in Statistical Modelling: A Comparative Study

Abstract

In this work, we present numerical methods appropriate for parameter estimation in high-dimensional statistical modelling. The solution of these problems is not unique and a crucial question arises regarding the way that a solution can be found. A common choice is to keep the corresponding solution with the minimum norm. There are cases in which this solution is not adequate and regularisation techniques have to be considered. We classify specific cases for which regularisation is required or not. We present a thorough comparison among existing methods for both estimating the coefficients of the model which corresponds to design matrices with correlated covariates and for variable selection for supersaturated designs. An extensive analysis for the properties of design matrices with correlated covariates is given. Numerical results for simulated and real data are presented.

Keywords

• high-dimensional
• minimum norm solution
• regularisation
• Tikhonov
• <i>ℓp</i>-<i>ℓq</i>
• variable selection