ESC-PAN: An Efficient CNN Architecture for Image Super-Resolution

Abstract

Deep Learning models, based on Convolutional Neural Network (CNN) architecture, have proven to be useful and effective in many image processing tasks, and have recently been shown to be effective for image Super-Resolution (SR). Common trends in SR improve the quality of the reconstructed image by increasing the depth and complexity of the CNN model. While this approach produces superior performance in objective image quality metrics (IQA), such as Peak-Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) index, having the number of parameters in the order of millions sacrifices the practicality of model deployment. This is especially true for applications that require real-time processing, such as online conferencing. In this paper, a CNN-based SR model architecture that integrates an attention mechanism while maintaining low complexity is proposed. The number of parameters of the model is reduced by adopting depthwise-separable convolution (DSC) throughout the model. Multiply-accumulate operations (MACs) are reduced by adopting a late upsampling scheme to operate only on low-dimensional features maps. Experimental results show that the proposed model architecture has better performance in terms of objective IQA metrics, such as PSNR and SSIM, and subjective IQA. This improved performance is achieved at a reduced complexity. We also showcase the scalability of the proposed CNN architecture by increasing the model complexity slightly to gain better desired performance.

Keywords

• Real-time image super-resolution
• depth-wise separable convolution
• self-calibrated convolution
• pixel attention
• image quality assessment