Self‐supervised image clustering from multiple incomplete views via constrastive complementary generation

Abstract

Abstract Incomplete Multi‐View Clustering aims to enhance clustering performance by using data from multiple modalities. Despite the fact that several approaches for studying this issue have been proposed, the following drawbacks still persist: (1) It is difficult to learn latent representations that account for complementarity yet consistency without using label information; (2) and thus fails to take full advantage of the hidden information in incomplete data results in suboptimal clustering performance when complete data is scarce. In this study, Contrastive Incomplete Multi‐View Image Clustering with Generative Adversarial Networks (CIMIC‐GAN), which uses Generative Adversarial Network (GAN) to fill in incomplete data and uses double contrastive learning to learn consistency on complete and incomplete data is proposed. More specifically, considering diversity and complementary information among multiple modalities, we incorporate autoencoding representation of complete and incomplete data into double contrastive learning to achieve learning consistency. Integrating GANs into the autoencoding process can not only take full advantage of new features of incomplete data, but also better generalise the model in the presence of high data missing rates. Experiments conducted on four extensively used data sets show that CIMIC‐GAN outperforms state‐of‐the‐art incomplete multi‐View clustering methods.

Keywords

• clustering from multiple incomplete views
• computer vision
• constrastive learning
• generative adversarial network