Class-Wise Adaptive Strategy for Semi Supervised Semantic Segmentation

Abstract

Semi-supervised semantic segmentation learns a model for classifying pixels into specific classes using a few labeled samples and numerous unlabeled images. The recent leading approach is consistency regularization by self-training with pseudo-labeling pixels having high confidences for unlabeled images. However, using only high-confidence pixels for self-training may result in losing much of the information in the unlabeled sets due to poor confidence calibration of modern deep learning networks. In this paper, we propose a class-wise adaptive strategy for semi supervised semantic segmentation (CASS) to cope with the loss of most information that occurs in existing high-confidence-based pseudo-labeling methods. Unlike existing semi-supervised semantic segmentation frameworks, CASS constructs a validation set on a labeled set, to leverage the calibration performance for each class. On this basis, we propose class-wise adaptive thresholds and class-wise adaptive over-sampling using the analysis results from the validation set. Our proposed CASS achieves state-of-the-art performance on the full data partition of the base PASCAL VOC 2012 dataset and on the 1/4 data partition of the Cityscapes dataset with significant margins of 83.0 and 80.4 mIoU, respectively.

Keywords

• Semi-supervised semantic segmentation
• over-sampling
• confidence threshold
• pseudo-labeling