Multiple Kernel Fuzzy Clustering With Unsupervised Random Forests Kernel and Matrix-Induced Regularization

Abstract

Although kernel fuzzy clustering can handle non-spherical clusters by mapping data to a more separable feature space, its performance is highly determined by the setting of kernels. So, the multiple kernel fuzzy clustering (MKFC) is proposed to obtain the flexibility in designing an optimal kernel from a large set of candidates. In MKFC, many predefined general kernels like Gaussian and polynomial ones are linearly aggregated and the weights of kernels are adjusted automatically. However, the performance of MKFC is greatly hampered by two noticeable problems. First, MKFC only uses predefined general kernels and pays less attention to the inherent structure of specific data. This leads to the trouble of selecting proper base kernels for different data. The second problem is the ignorance of correlations between kernels in MKFC. It results in redundant kernels being used to define the feature space. This paper solves the two problems simultaneously by introducing a new MKFC model. Based on unsupervised random forests (RFs), some data-dependent kernels are generated and combined with others to build a more representative feature space. The correlations between kernels are also calculated and inserted into the objective function of fuzzy clustering as a matrix-induced regularization to encourage the diversity in kernels. We name the new model as MKFC with unsupervised RFs kernel and matrix-induced regularization. The optimization algorithm for the new model is derived, and the experiments on benchmark datasets demonstrate its superiority over other MKFC approaches.

Keywords

• Fuzzy clustering
• multiple kernel
• unsupervised random forests
• regularization