Global–Local Query-Support Cross-Attention for Few-Shot Semantic Segmentation

Abstract

Few-shot semantic segmentation (FSS) models aim to segment unseen target objects in a query image with scarce annotated support samples. This challenging task requires an effective utilization of support information contained in the limited support set. However, the majority of existing FSS methods either compressed support features into several prototype vectors or constructed pixel-wise support-query correlations to guide the segmentation, which failed in effectively utilizing the support information from the global–local perspective. In this paper, we propose Global–Local Query-Support Cross-Attention (GLQSCA), where both global semantics and local details are exploited. Implemented with multi-head attention in a transformer architecture, GLQSCA treats every query pixel as a token, aggregates the segmentation label from the support mask values (weighted by the similarities with all foreground prototypes (global information)), and supports pixels (local information). Experiments show that our GLQSCA significantly surpasses state-of-the-art methods on the standard FSS benchmarks PASCAL-5i and COCO-20i.

Keywords

• few-shot semantic segmentation
• global–local query-support cross-attention
• multi-head attention
• transformer