No Reference Pansharpened Image Quality Assessment Through Deep Feature Similarity

Abstract

Pansharpening refers to the process of enhancing the spatial resolution of a multispectral image with the help of a high spatial resolution panchromatic (PAN) image. Quality assessment (QA) of pansharpened images helps provide a formal framework for the analysis and design of pansharpening methods and is thus extremely important. However, lack of availability of a reference multispectral image makes QA of pansharpening algorithms a challenging task. Given the popular use of QA algorithms that use a reference, this article focuses on predicting the quality under a “no-reference” (NR) setting. Specifically, a learning based NR pansharpened image quality assessment (IQA) approach is adopted to predict state-of-the-art reference-based measures such as $Q2^{n}$ and spectral angle mapper without the need of a reference. We design an end-to-end deep pansharpening IQA network to compute the similarity of deep features fused from the PAN and input low-resolution multispectral with similar features extracted from the given pansharpened image. To train and test our learning-based approach, we create a large corpus of pansharpened images belonging to different satellites and thematic scenes by applying different pansharpening algorithms. Our experiments demonstrate that our NR pansharpened IQA algorithm achieves excellent performance and generalizes well across different satellites and resolutions.

Keywords

• Deep learning
• image fusion
• pansharpening
• quality assessment (QA)