A Simple Framework of Few-Shot Learning Using Sparse Annotations for Semantic Segmentation of 3-D Point Clouds

Abstract

The semantic segmentation of point clouds plays a crucial role in the interpretation of 3-D scene. However, the majority of supervised learning methods needs a great number of annotated data to train an effective model, which requires labor-intensive and time-consuming annotation of 3-D points. In this work, we presented a simple but practical framework that adopts a few-shot learning strategy to semantically segment 3-D point clouds using merely sparse annotations. Major contributions lie in two major aspects. 1) A few-shot learning framework for semantic segmentation with sparse annotation is designed, which requires less manual labeling work than conventional methods. 2) A distribution calibration method is introduced to achieve transfer learning based on pretrained models and requires no retraining of neural networks. The accuracy of semantic segmentation has been greatly boosted by adapting the model trained with a small number of annotated data to a wide range of data through synchronous transfer learning. The proposed method is tested by two groups of benchmarks, namely the combination of the TUM-MLS-2016 dataset and the Toronto-3D MLS dataset and the combination of the DALES dataset and the ISPRS benchmark dataset. We have obtained impressive results reaching the same level as the results of the state-of-the-art method using dense annotation with an overall accuracy of 0.81 for segmenting eight semantic classes on the Toronto-3D MLS dataset, and an overall accuracy of 0.81 for segmenting nine semantic classes on the ISPRS benchmark dataset, proving the effectiveness of the proposed framework.

Keywords

• Distribution calibration
• mobile laser scanning (MLS)
• semantic segmentation
• sparse annotation