Robust Semi-Supervised Non-Negative Matrix Factorization With Structured Normalization

Abstract

Non-negative matrix factorization (NMF) approximates a non-negative data matrix with the product of two low-rank non-negative matrices by minimizing the cost of such approximation. However, traditional NMF models cannot be generalized in the cases when the dataset contains outliers and limited knowledge from domain experts. In this paper, we propose a robust semi-supervised NMF model (RSS-NMF) to overcome the aforementioned deficiency. RSS-NMF utilizes the L2/L1-norm to encourage approximation and makes the model insensitive to outliers by prohibiting them from dominating the cost function. To incorporate the discriminative information, RSS-NMF utilizes the structured normalization method when learns a diagonal matrix to normalize the coefficients such that they get close to the label indicators of the given labeled examples. Although the multiplicative update rule (MUR) can be adopted to minimize RSS-NMF, it converges slowly. In this paper, we adopt a fast gradient descent algorithm (FGD) to optimize RSS-NMF and prove its convergence to a stationary point. FGD uses a Newton method to search the optimal step length and thus, FGD converges faster than MUR. The experimental results show the promise of RSS-NMF comparing with the representative clustering models on several face image datasets.

Keywords

• Non-negative matrix factorization
• semi-supervised learning
• L₂/L₁-norm%22%2C%22default_operator%22%3A%22AND%22%7D%7D%7D"> <italic xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">L</italic>₂/<italic xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">L</italic>₁-norm
• structured normalization