DeepInterpolation: fusion of multiple interpolations and CNN to obtain super‐resolution

Abstract

The authors propose architectures that learn end‐to‐end mapping functions to improve the spatial resolution of the input natural images. The models are unique in forming non‐linear combinations of three image interpolation techniques using the convolutional neural network. Another proposed architecture uses a skip connection with nearest‐neighbour interpolation, achieving almost similar results. The architectures have been carefully designed to ensure that the reconstructed images lie precisely in the manifold of high‐resolution images, thereby preserving the high‐frequency components with fine details. They have compared with the state‐of‐the‐art and recent deep learning‐based natural image super‐resolution techniques and found that their methods can preserve the sharp details in the image, while also obtaining comparable or better peak‐signal‐to‐noise ratio values than them. Since their methods use image interpolations and a shallow convolutional neural network (CNN) with a fewer number of smaller filters, the computational cost is kept low. They have reported the results of the best two proposed architectures on five standard data sets for an upscale factor of 2. Their methods generalise well in most cases, which is evident from the better results obtained with increasingly complex data sets. For four times upscaling, they have designed similar architectures for comparing with other methods.

Keywords

• image interpolation techniques
• skip connection
• nearest‐neighbour interpolation
• reconstructed images
• high‐resolution images
• high‐frequency components