Hybrid Attention Mechanism and Forward Feedback Unit for RGB-D Salient Object Detection

Abstract

RGB-D saliency object detection (SOD) is an important pre-processing operation for various computer vision tasks and has received much attention in recent years. However, how to extract more effective features and how to effectively fuse RGB and depth modality features are still challenges that restrict the development of SOD. In this paper, we propose an effective network architecture called FFMA-Net: 1) We replace the backbone network of the baseline with a ResNet34 model to extract more effective features from the input data; 2) We design the HAM module to refine the features extracted by the ResNet34 model at different stages to ensure the effectiveness of features from each stage; 3) We propose the FFU module to perform multi-scale fusion of features from different stages, resulting in more semantic-rich features that are crucial for the decoding stage of the model. Finally, our model performs better than the latest methods on six RGB-D datasets on all evaluation metrics, especially in terms of F-measure metric, which shows significant improvement with approximately 5% on both SSD and LFSD datasets.

Keywords

• RGB-D salient object detection
• forward feedback unit
• hybrid attention mechanism