DIAL: Deep Interactive and Active Learning for Semantic Segmentation in Remote Sensing

Abstract

In this article, we propose to build up a collaboration between a deep neural network and a human in the loop to swiftly obtain accurate segmentation maps of remote sensing images. In a nutshell, the agent iteratively interacts with the network to correct its initially flawed predictions. Concretely, these interactions are annotations representing the semantic labels. Our methodological contribution is twofold. First, we propose two interactive learning schemes to integrate user inputs into deep neural networks. The first one concatenates the annotations with the other network’s inputs. The second one uses the annotations as a sparse ground truth to retrain the network. Second, we propose an active learning (AL) strategy to guide the user toward the most relevant areas to annotate. To this purpose, we compare different state-of-the-art acquisition functions to evaluate the neural network uncertainty such as ConfidNet, entropy, or ODIN. Through experiments on three remote sensing datasets, we show the effectiveness of the proposed methods. Notably, we show that AL based on uncertainty estimation enables to quickly lead the user toward mistakes and that it is thus relevant to guide the user interventions. Code will be open-source and released in this repository.1

Keywords

• Active learning (AL)
• deep learning
• earth observation
• interactive segmentation
• semantic segmentation