Adaptive Coarse-to-Fine Single Pixel Imaging With Generative Adversarial Network Based Reconstruction

Abstract

Single Pixel Imaging (SPI) that only uses one light intensity sensor has been researched extensively as an alternative imaging method. It is proven to work at low light conditions and unconventional wavelength bands where the classical pixel array sensors are limited. However, the major issues of SPI remain as the image quality and computational efficiency. Thus, this paper proposes an adaptive coarse-to-fine (C2F) sampling method to replace the typical uniform sampling method to achieve image reconstruction with better quality. This scalable sampling mechanism is adaptive to the target scene as it will progressively sample according to the image complexity and quality indicator. Subsequently, a deep Generative Adversarial Network (GAN) model is also proposed to improve the time efficiency of the multi-scale image reconstruction. The results show that C2F sampling consistently outperforms uniform sampling in terms of image quality (21% in SSIM, 8% in PSNR and 17% in RMSE). Besides, improvement in the efficiency is also achieved by the proposed GAN reconstruction, whereby the total time taken is only 0.025% of the time taken for the conventional L1 reconstruction method ( $\approx 4000$ times faster). In conclusion, the proposed adaptive C2F SPI using GAN reconstruction method can serve as an optimised solution to improve both the image quality and computational efficiency in SPI.

Keywords

• Coarse-to-fine
• compressed sensing
• deep learning
• GAN
• single pixel imaging