Parallax‐based second‐order mixed attention for stereo image super‐resolution

Abstract

Abstract Stereo image pairs can effectively enhance the performance of super‐resolution (SR) since both intra‐view and cross‐view information can be used. However, exploiting cross‐view information accurately is extremely challenging. Most recent methods use the attention mechanism to get stereo correspondence. But these methods ignore the high‐frequency information since they only utilise first‐order statistics, which leads to reducing the discriminative ability of the network. To address this issue, in this work, a parallax‐based second‐order mixed attention stereo SR network (PSMASSRnet) is proposed to integrate the cross‐view information from a stereo image pair for SR. Specifically, a novel parallax‐based second‐order mixed attention module (PSMAM) is developed to combine second‐order channel features and spatial features to obtain more discriminative representations. Furthermore, a dense cross‐atrous spatial pyramid pooling (ASPP) module is also presented, which can effectively explore the local and the multi‐scale features with different dilation rates to extract more discriminative features with fewer parameters and less execution. The extensive experiments on the KITTI2012, KITTI2015 and Middlebury datasets have demonstrated the superiority of the proposed PSMASSRnet over the state‐of‐the‐art methods in the aspects of both the quantitative metrics and the visual quality.

Keywords

• stereo image processing
• image resolution
• feature extraction
• image representation
• statistical analysis