Minimum Residual Sum of Squares Estimation Method for High-Dimensional Partial Correlation Coefficient

Abstract

The partial correlation coefficient (Pcor) is a vital statistical tool employed across various scientific domains to decipher intricate relationships and reveal inherent mechanisms. However, existing methods for estimating Pcor often overlook its accurate calculation. In response, this paper introduces a minimum residual sum of squares Pcor estimation method (MRSS), a high-precision approach tailored for high-dimensional scenarios. Notably, the MRSS algorithm reduces the estimation bias encountered with positive Pcor. Through simulations on high-dimensional data, encompassing both sparse and non-sparse conditions, MRSS consistently mitigates the arithmetic bias for positive Pcors, surpassing other algorithms discussed. For instance, for large sample sizes (n≥100) with Pcor > 0, the MRSS algorithm reduces the MSE and RMSE by about 30–70% compared to other algorithms. The robustness and stability of the MRSS algorithm is demonstrated by the sensitivity analysis with variance and sparsity parameters. Stocks data in China’s A-share market are employed to showcase the MRSS methodology’s practicality.

Keywords

• partial correlation
• high-dimensional data
• variable selection
• MCP