Practical debiasing with the Covariant Prior in the proportional regime when p < n

Abstract

We show that the Covariant Prior can be used to effectively de-bias the resulting MAP estimator in the proportional regime, where the number of covariates p grows proportionally to n , the number of samples, with p < n . The asymptotics of the resulting MAP estimator can be studied via the statistical physics approach. It is known that the Replica method leads to three equations for three scalar order parameters. Knowledge of the order parameters and of the true signal strength, allows the computation of the asymptotic bias. Here we show that all these quantities might be estimated from the data alone, without actually solving the equations of the theory (which require the knowledge of the data generating process). Once this is done, a de-biased estimator can be computed easily allowing for eventual testing. We emphasize that the proposed methodology does not require estimating the true covariance matrix, and can be (easily) applied whenever the latter is positive definite.

Keywords

• high dimensional regression
• replica method
• de-biasing