Simulation of Multispectral and Hyperspectral EO Products for Onboard Machine Learning Application

Abstract

Over the past decade, there has been a rapid acceleration in the development of Artificial Intelligence (AI) for Earth Observation (EO), driven by the exponential growth in collected data and advances in algorithms and computing. This revolution extends to onboard EO spacecraft, enabling the development of smart satellites and software-defined missions with more autonomy and reconfigurability. This article discusses the challenges linked to data quantity, diversity, accountability, and onboard representativeness for the implementation of deep neural networks onboard EO missions. To address these challenges, the article introduces a new framework for simulating radiance and reflectance for future hyperspectral or multispectral imaging satellites based on existing missions. This framework is modular and versatile, allowing for the generation of synthetic images across various optical sensors and environmental conditions. The effectiveness and flexibility of the approach is demonstrated by applying it to two different missions, $\Phi$sat-2 and IMAGIN-e. Its practical deployment and effectiveness in real-world scenarios were demonstrated via an open community challenge organized in 2023 and dubbed OrbitalAI. With over a hundred registered teams, a set of 15 teams was preselected, proposing a wide range of AI applications trained via the proposed framework. The EO use cases encompass the monitoring of flood, wildfire, build-up area, water quality, or crops. In addition, comprehensive datasets of simulated images are created and shared, representing up to 520 GB and providing a significant resource for developing AI algorithms for space applications. Both the simulation framework and datasets are made open-source to foster further research in this domain.

Keywords

• $\Phi$ sat-2%22%2C%22default_operator%22%3A%22AND%22%7D%7D%7D"> <named-content xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" content-type="math" xlink:type="simple"> <inline-formula> <tex-math notation="LaTeX">$\Phi$</tex-math> </inline-formula> </named-content>sat-2
• atmospheric correction
• cloud detection
• deep learning
• onboard computing
• reflectance