Semantic-Edge Interactive Network for Salient Object Detection in Optical Remote Sensing Images

Abstract

Despite salient object detection in natural images has made remarkable progress, it is still an emerging and challenging problem to detect salient objects from optical remote sensing images [remote sensing image salient object detection (RSI-SOD)]. To improve RSI-SOD based on fully convolutional networks (FCNs), attention and edge awareness have been used separately to aid integration and refinement of multilevel features for effective decoding. Although they have been shown to semantically enhance salient features and reduce fuzzy boundaries, the correlation between the semantic-enhanced salient features and edge features is rarely explored, which has inspired the development of a new model to enable close interaction between semantic and edges for fully activating the advantages of attention and edge awareness, and led to the semantic-edge interactive network (SEINet) presented in this article. The proposed model consists of two interacting decoding branches based on the U-shaped network to achieve salient object detection (SOD) and salient edge detection (SED), and the multiscale attention interaction (MAI) module is proposed to provide edge-enhanced semantic for SOD and semantic-enhanced edge for SED interactively between the two branches. Moreover, to alleviate the problem of semantic dilution, the semantic-guided fusion (SF) module is proposed and deployed at the end of the SOD branch. From the extensive quantitative and qualitative comparison of the proposed model against the FCN-based models with and without incorporation of attention and edge awareness, the proposed model obtains the most stable scores at different thresholds of the $F$-measure curve and outperforms 18 state-of-the-art methods.

Keywords

• Attention mechanism
• edge-aware
• multitask learning
• optical remote sensing image (RSI)
• salient object detection (SOD)