Robust Hierarchical Learning for Non-Negative Matrix Factorization With Outliers

Abstract

Desirable properties of extensions of non-negative matrix factorization (NMF) include robustness in the presence of noises and outliers, ease of implementation, the guarantee of convergence, operation in an automatic fashion that trades off the balance between data approximation and model simplicity well, and the capability to model the inherently sequential structure of time-series signals. The state-of-the-art methods typically have only a subset of these aforementioned properties and seldom simultaneously possess them all. In this paper, we propose a novel approach that provides all these desirable properties by extending the automatic relevance determination framework in NMF from Tan and Févotte. Starting from an objective function derived from the maximum a posterior estimation of a Bayesian model, we develop majorization-minimization algorithms that work effectively to determine the correct model order, regardless of the impact of noise and outliers. Subsequently, we give a rigorous convergence analysis of the proposed algorithms. Moreover, convolutive bases are also incorporated in the basic model so that it is able to capture the richness of temporal continuity. We perform experiments on both synthetic and real-world data sets to show the efficiency and robustness of our approach.

Keywords

• Non-negative matrix factorization
• automatic relevance determination
• majorization-minimization
• convolutive bases