Self-supervised Deep Clustering Algorithm Based on Self-attention

Abstract

In recent years,deep clustering methods using joint optimization strategy,such as DEC (deep embedding clustering) and DDC (deep denoising clustering) algorithms,have made great progress in image clustering that heavily related to features representation ability of deep networks,and brought certain degree breakthroughs in clustering performances.The quality of feature extraction directlyaffects the subsequent clustering tasks.However,the generalization abilities of these methods are not satisfied,exactly as different network structures are used in different datasets to guarantee the clustering performance.In addition,there is a quite larger space to enhance clustering performances compared to classification performances.To this end,a self-supervised deep clustering (SADC) method based on self-attention is proposed.Firstly,a deep convolutional autoencoder is designed to extract features,and noisy images are employed to enhance the robustness of the network.Secondly,self-attention mechanism is combined with the proposed network to capture useful features for clustering.At last,the trained encoder combines with K-means algorithm to form a deep clustering model for feature representation and clustering assignment,and iteratively updates parameters to improve the clustering accuracy and generalization ability of the proposed network.The proposed clustering method is verified on 6 traditional image datasets and compared with the deep clustering algorithms DEC and DDC.Experimental results show that the proposed SADC can provide better clustering results,and is comparable to the state-of-the-art clustering algorithms.Overall,the unified network structure ensures the clustering accuracy and simultaneously reducing computational complexity of the deep clustering algorithms.

Keywords

• deep convolutional autoencoder|image clustering|features representation|self-attention|computational complexity