An RMT-based generalized Bayesian information criterion for signal enumeration

Abstract

Abstract This paper provides a method for enumerating signals impinging on an array of sensors based on the generalized Bayesian information criterion (GBIC). The proposed method motivates by a statistic for testing the sphericity of the covariance matrix when the sample size n is less than the dimension m. The statistic consists of the first four moments of sample eigenvalue distribution and relaxes the assumption of Gaussian distribution. We derive the asymptotical distribution of the statistic as m, n tends to infinity at the same ratio by random matrix theory and propose the expression of GBIC for determining the signal number. Numerical simulations demonstrate that the proposed method has a high probability of detection in both the Gaussian and the non-Gaussian noise, and performs better than other methods.

Keywords

• Signal enumeration
• Generalized Bayesian information criterion
• Sphericity test
• Random matrix theory