Multi-Modal Subspace Fusion via Cauchy Multi-Set Canonical Correlations

Abstract

Multi-set canonical correlation analysis (MCCA) is a famous multi-modal coherent subspace learning method. However, sample-based between-modal and within-modal covariance matrices of MCCA usually deviate from real covariance matrices due to noise information and limited sample size. The deviation will weaken the performance of MCCA, especially in image recognition. Aiming at this challenging issue, we correct singular values of sample covariance matrices with the employment of Cauchy estimate theory and further obtain Cauchy covariance matrices that are closer to real covariance matrices. On the basis of Cauchy covariance matrices, we develop a novel multi-modal subspace fusion method, i.e. Cauchy multi-set canonical correlations. By maximizing Cauchy correlations between different modalities and constraining Cauchy scatters of within-modal data, the method can learn a Cauchy coherent fusion subspace with well discriminative power from a few images. Experiment results have shown the effectiveness of the proposed method, promising to the aims of this research.

Keywords

• Multi-modal subspace fusion
• coherent fusion subspace learning
• correlation projection theory
• image recognition