Neural Network Emulation of Synthetic Hyperspectral Sentinel-2-Like Imagery With Uncertainty

Abstract

Hyperspectral satellite imagery provides highly resolved spectral information for large areas and can provide vital information. However, only a few imaging spectrometer missions are currently in operation. Aiming to generate synthetic satellite-based hyperspectral imagery potentially covering any region, we explored the possibility of applying statistical learning, i.e., emulation. Based on the relationship of a Sentinel-2 (S2) scene and a hyperspectral HyPlant airborne image, this work demonstrates the possibility to emulate a hyperspectral S2-like image. We tested the role of different machine learning regression algorithms and varied the image-extracted training dataset size. We found superior performance of neural network as opposed to the other algorithms when trained with large datasets (up to 100 000 samples). The developed emulator was then applied to the L2A (bottom-of-atmosphere reflectance) S2 subset, and the obtained S2-like hyperspectral reflectance scene was evaluated. The validation of emulated against reference spectra demonstrated the potential of the technique. $R^{2}$ values between 0.75 and 0.9 and NRMSE between 2 and 5% across the full 402–2356 nm range were obtained. Moreover, epistemic uncertainty is obtained using the dropout technique, revealing spatial fidelity of the emulated scene. We obtained highest SD values of 0.05 (CV of 8%) in clouds and values below 0.01 (CV of 7%) in vegetation land covers. Finally, the emulator was applied to an entire S2 tile (5490 × 5490 pixels) to generate a hyperspectral reflectance datacube with the texture of S2 (60 Gb, at a speed of 0.14 s/10000 pixels). As the emulator can convert any S2 tile into a hyperspectral image, such scenes give perspectives how future satellite imaging spectroscopy will look like.

Keywords

• Emulation
• hyplant
• hyperspectral
• neural networks (NNs)
• Sentinel-2 (S2)