Novel View Synthesis and Dataset Augmentation for Hyperspectral Data Using NeRF

Abstract

Hyperspectral data for the 3D domain is relatively difficult to acquire. Existing hyperspectral datasets are unsuitable for 3D research, suffer from issues of severe data scarcity, and a lack of multi-perspective images of the same object, etc. To address these challenges, data augmentation with limited data is essential. In this study, we applied neural rendering method (such as Neural Radiance Field) to hyperspectral images for dataset augmentation. We conducted experiments on novel view synthesis for hyperspectral images from 360-degree multi-perspectives, demonstrating that our method can generate high-quality hyperspectral images from various perspectives. Through experiments involving key points extraction and 3D reconstruction, we validated the efficacy of generating a substantial volume of high-quality hyperspectral images from a restricted set of varying perspectives. These results contribute to addressing the challenges associated with data augmentation. We also conducted experiments of neural radiance fields in the hyperspectral data domain under different network parameters and training conditions to find the appropriate settings.

Keywords

• Dataset augmentation
• hyperspectral image
• NeRF
• novel view synthesis