Class-Variational Learning With Capsule Networks for Deep Entity-Subspace Clustering

Abstract

The progression of deep clustering techniques in the recent years emphasizes the need for unsupervised representation learning methods that build lower-dimensional embeddings within expressive latent feature spaces. An important performance factor for such techniques constitutes the representational capacity of the used neural network technology. Although Capsule Networks (CapsNet)s are predestinated for the task of deep clustering through their rich entity representations and inter-layer dynamics related to clustering, CapsNets are to date rarely explored in this context. The main challenge for enabling unsupervised representation learning with CapsNets results from the required differentiation of the output capsules to encompass data-intrinsic classes. This paper proposes a novel end-to-end framework denominated as Class-Variational Learning (CVL) which utilizes an asymmetric autoencoder consisting of a CapsNet encoder and a non-capsular decoder network for facilitating entity-subspace clustering. To the best of our knowledge CVL represents the first approach which accomplishes a class-to-capsule specialization of the output capsules without external supervisory signals. As unique characteristic, CVL forms an equivariant latent space with continuous transitions between data-intrinsic classes. This means a crucial gain in the explainability of the constructed inference mechanism, since the class-discriminative equivariant space is linearly navigable and fully accessible by a human actor. Despite our CVL model does currently not lead to competitive accuracies compared to the state-of-the-art deep clustering techniques, CVL opens promising perspectives on the use of CapsNets as the basis for deep clustering which hopefully motivates future research in this field.

Keywords

• Autoencoder
• capsule networks
• class-discriminative equivariance
• deep clustering
• unsupervised representation learning