A Lightweight Recurrent Aggregation Network for Satellite Video Super-Resolution

Abstract

Intelligent processing and analysis of satellite video has become one of the research hotspots in the representation of remote sensing, and satellite video super-resolution (SVSR) is an important research direction, which can improve the image quality of satellite video. However, existing approaches for SVSR often underutilize a notable advantage inherent to satellite video, the presence of extensive sequential imagery capturing a consistent scene. Presently, the majority of SVSR methods merely harness a limited number of adjacent frames for enhancing the resolution of individual frames, thus resulting in suboptimal information utilization. In response, we introduce the recurrent aggregation network for satellite video superresolution (RASVSR). This innovative framework leverages a bidirectional recurrent neural network to propagate extracted features from each frame across the entire video sequence. It relies on an alignment method based on optical flow and deformable convolution (DCN) to realize the alignment of the features, and a temporal feature fusion module to realize effective feature fusion over time. Notably, our research underscores the positive influence of employing lengthier image sequences in SVSR. In the context of RASVSR, with better alignment and fusion, we make the perceptual field of each frame spanning 100 frames of the video, thus, acquiring richer information, and information between different images can be complementary. This strategic approach culminates in superior performance compared with alternative methods, as evidenced by a noteworthy 1.15 dB improvement in PSNR, with very few parameters.

Keywords

• Attention mechanism
• recurrent neural network (RNN)
• satellite video
• video super-resolution (VSR)