Fuzzy Divergence Weighted Ensemble Clustering With Spectral Learning Based on Random Projections for Big Data

Abstract

In many real-world applications, data are described by high-dimensional feature spaces, posing new challenges for current ensemble clustering methods. The goal is to combine sets of base clusters to enhance clustering accuracy, but this makes them susceptible to low quality. However, the reliability of present ensemble clustering in high-dimensional data still needs improvement. In this context, we propose a new fuzzy divergence-weighted ensemble clustering based on random projection and spectral learning. Firstly, random projection (RP) is used to create various dimensional data and find membership matrices via fuzzy c-means (FCM). Secondly, fuzzy partitions of random projections are ranked using entropy-based local weighting along with Kullback-Leibler (KL) divergence to detect any uncertainty. Then it used to evaluate the weight of each cluster. Finally, we create regularized graphs from these membership matrices and use spectral matrices to estimate the affinity matrices of these graphs using fuzzy KL divergence anchor graphs. Subsequently, obtaining the final clustering results is considered as an optimization problem, and the ensemble clustering results are obtained. The experimental results on high-dimensional data demonstrate the efficiency of our method compared to state-of-the-art methods.

Keywords

• Fuzzy ensemble clustering
• high-dimensional data
• random projection
• Kullback-Leibler divergence entropy
• spectral learning