Survey on Image Panoptic Segmentation Based on Deep Learning

Abstract

With the continuous development of deep learning and image segmentation, image panoptic segmentation has become a research hotspot in the field of computer vision, and many image panoptic segmentation methods have been proposed. This paper summarizes the research methods of image panoptic segmentation based on deep learning. Firstly, the research status of image panoptic segmentation at home and abroad is introduced, and the existing image panoptic segmentation methods are classified according to different optimization tasks in the network architecture, mainly including image panoptic segmentation optimized by feature extraction, image panoptic segmentation optimized by sub-task segmentation, image panoptic segmentation optimized by sub-task fusion, and other image panoptic segmentation. Secondly, 5 commonly used datasets, i.e. MS COCO, PASCAL VOC, Cityscapes, ADE20K and Mapillary Vistas,  and 2 evaluation criteria, i.e. panoptic quality (PQ) and parsing covering (PC) in image panoptic segmentation are briefly introduced. And then, performance comparison of typical image panoptic segmentation methods has been conducted on different datasets. Thirdly, the application of image panoptic segmentation in medicine, autonomous driving, drones, agriculture, animal husbandry, military and other fields are listed. Finally, the deficiencies and challenges of existing methods in complex scene applications, real-time performance, and conflicts are pointed out, and the potential directions of image panoptic segmentation are discussed, including image panoptic segmentation based on a simple unified framework, real-time high-quality image panoptic segmentation, and image panoptic segmentation in complex application scenarios.

Keywords

• image panoptic segmentation; deep learning; feature extraction; sub-task segmentation; sub-task fusion