Attention-Based Joint Semantic-Instance Segmentation of 3D Point Clouds

Abstract

In this paper, we propose an attention-based instance and semantic segmentation joint approach, termed ABJNet, for addressing the instance and semantic segmentation of 3D point clouds simultaneously. First, a point feature enrichment (PFE) module is used to enrich the segmentation network’s input data by indicating the relative importance of each point’s neighbors. Then, a more efficient attention pooling operation is designed to establish a novel module for extracting point cloud features. Finally, an efficient attention-based joint segmentation module (ABJS) is proposed for combining semantic features and instance features in order to improve both segmentation tasks. We evaluate the proposed attention-based joint semantic-instance segmentation neural network (ABJNet) on a variety of indoor scene datasets, including S3DIS and ScanNet V2. Experimental results demonstrate that our method outperforms the start-of-the-art method in 3D instance segmentation and significantly outperforms it in 3D semantic segmentation.

Keywords

• computer graphics
• object segmentation
• feature extraction
• pattern recognition
• machine learning