Multi‐similarity based hyperrelation network for few‐shot segmentation

Abstract

Abstract Few‐shot semantic segmentation aims at recognizing the object regions of unseen categories with only a few annotated examples as supervision. The key to few‐shot segmentation is to establish a robust semantic relationship between the support and query images and to prevent overfitting. In this paper, an effective multi‐similarity hyperrelation network (MSHNet) is proposed to tackle the few‐shot semantic segmentation problem. In MSHNet, a new generative prototype similarity (GPS) is proposed, which, together with cosine similarity, establishes a strong semantic relationship between supported images and query images. In addition, a symmetric merging block (SMB) in MSHNet is proposed to efficiently merge multi‐layer, multi‐shot, multi‐similarity features to generate hyperrelation features for semantic segmentation. Experimenting on two benchmark semantic segmentation datasets (Pascal − 5i and COCO − 20i) shows that this method achieves a mean intersection‐over‐union score of 72.3% and 56.0%, respectively, which outperforms the state‐of‐the‐art methods by 1.9% and 6.5%.