Deep Joint Demosaicking and Super-Resolution for Spectral Filter Array Images

Abstract

In spectral imaging, the constraints imposed by hardware often lead to a limited spatial resolution within spectral filter array images. On the other hand, the process of demosaicking is challenging due to intricate filter patterns and a strong spectral cross correlation. Moreover, demosaicking and super resolution are usually approached independently, overlooking the potential advantages of a joint solution. To this end, we use a two-branch framework, namely a pseudo-panchromatic image network and a pre-demosaicking sub-branch coupled with a novel deep residual demosaicking and super resolution module. This holistic approach ensures a more coherent and optimized restoration process, mitigating the risk of error accumulation and preserving image quality throughout the reconstruction pipeline. Our experimental results underscore the efficacy of the proposed network, showcasing an improvement of performance both qualitatively and quantitatively when compared to the sequential combination of state-of-the-art demosaicking and super resolution. With our proposed method, we obtained on the ARAD-1K dataset an average PSNR of 48.02 (dB) for domosaicking only, equivalent to the best method of the state-of-the-art. Moreover, for joint demosaicking and super resolution our model averages 35.26 (dB) and 26.29 (dB), respectively for $\times 2$ and $\times 4$ upscale, outperforming state-of-the-art sequential approach.The codes and datasets are available at https://github.com/HamidFsian/DRDmSR.

Keywords

• Spectral imaging
• demosaicking
• super resolution
• deep learning
• pseudo-panchromatic image
• spectral filter array