Remote Sensing (Apr 2022)

LiDAR-Based Real-Time Panoptic Segmentation via Spatiotemporal Sequential Data Fusion

  • Weiqi Wang,
  • Xiong You,
  • Jian Yang,
  • Mingzhan Su,
  • Lantian Zhang,
  • Zhenkai Yang,
  • Yingcai Kuang

DOI
https://doi.org/10.3390/rs14081775
Journal volume & issue
Vol. 14, no. 8
p. 1775

Abstract

Read online

Fast and accurate semantic scene understanding is essential for mobile robots to operate in complex environments. An emerging research topic, panoptic segmentation, serves such a purpose by performing the tasks of semantic segmentation and instance segmentation in a unified framework. To improve the performance of LiDAR-based real-time panoptic segmentation, this study proposes a spatiotemporal sequential data fusion strategy that fused points in “thing classes” based on accurate data statistics. The data fusion strategy could increase the proportion of valuable data in unbalanced datasets, and thus managed to mitigate the adverse impact of class imbalance in the limited training data. Subsequently, by improving the codec network, the multiscale features shared by semantic and instance branches were efficiently aggregated to achieve accurate panoptic segmentation for each LiDAR scan. Experiments on the publicly available dataset SemanticKITTI showed that our approach could achieve an effective balance between accuracy and efficiency, and it was also applicable to other point cloud segmentation tasks.

Keywords