Autonomous novel class discovery for vision-based recognition in non-interactive environments

Abstract

Visual recognition with deep learning has recently been shown to be effective in robotic vision. However, these algorithms tend to be build under fixed and structured environment, which is rarely the case in real life. When facing unknown objects, avoidance or human interactions are required, which may miss critical objects or be prohibitively costly to obtain on robots in the real world. We consider a practical problem setting that aims to allow robots to automatically discover novel classes with only labelled known class samples in hand, defined as open-set clustering (OSC). To address the OSC problem, we propose a framework combining three approaches: 1) using selfsupervised vision transformers to mitigate the discard of information needed for clustering unknown classes; 2) adaptive weighting for image patches to prioritize patches with richer textures; and 3) incorporating a temperature scaling strategy to generate more separable feature embeddings for clustering. We demonstrate the efficacy of our approach in six fine-grained image datasets.

Keywords

• Open set
• Image recognition
• Image clustering
• Deep learning
• Deep neural networks