Multi-scale YOLACT for instance segmentation

Abstract

The mainstream instance segmentation is a comprehensive computer vision task, which involves computer vision tasks such as image classification, object detection, and semantic segmentation. Aiming at the Prototype mask for initial segmentation mask with incorrect segmentation, this paper uses YOLACT (You Only Look at CoefficienTs) as the benchmark, in order to improve the network performance in the interfernce situation by enhancing the response of prototype mask, the multi-scale YOLACT for instance segmentation (MS YOLACT) is proposed, which increases the accuracy of segmentation by designing a lightweight network structure. First, the image gets multi-scale features through the residual network and the feature pyramid network. Then, the deep up-sampling and shallow down-sampling in the multi-scale feature layer are realized respectively to the size required by the prototype mask branch input, and all the deep information that has been up-sampled is further learned by convolution. Finally, at the input end of the prototype mask branch, the deep information and the shallow information are sequentially merged in an additive manner to improve the response of the prototype mask, thereby improving the accuracy of the target's mask segmentation. The experimental results show that compared with the benchmark on COCO test-dev, when the speed is reduced by only 1 FPS, the overall segmentation accuracy reaches an improvement of 0.6, and the segmentation accuracy of small and large targets reaches an improvement of 0.4 and 0.7 respectively; the visualization results also show that the segmentation mask of MS YOLACT is more accurate. In addition, MS YOLACT has the advantages of higher speed and lower requirements on equipment.

Keywords

• Instance segmentation
• The response of prototype mask
• Multi-scale features
• Semantic information
• Detailed information