A deep learning framework based on generative adversarial networks and vision transformer for complex wetland classification using limited training samples

Abstract

Wetlands have long been recognized among the most critical ecosystems globally, yet their numbers quickly diminish due to human activities and climate change. Thus, large-scale wetland monitoring is essential to provide efficient spatial and temporal insights for resource management and conservation plans. However, the main challenge is the lack of enough reference data for accurate large-scale wetland mapping. As such, the main objective of this study was to investigate the efficient deep-learning models for generating high-resolution and temporally rich training datasets for wetland mapping. The Sentinel-1 and Sentinel-2 satellites from the European Copernicus program deliver radar and optical data at a high temporal and spatial resolution. These Earth observations provide a unique source of information for more precise wetland mapping from space. The second objective was to investigate the efficiency of vision transformers for complex landscape mapping. As such, we proposed a 3D Generative Adversarial Network (3D GAN) to best achieve these two objectives of synthesizing training data and a Vision Transformer model for large-scale wetland classification. The proposed approach was tested in three different study areas of Saint John, Sussex, and Fredericton, New Brunswick, Canada. The results showed the ability of the 3D GAN to stimulate and increase the number of training data and, as a result, increase the accuracy of wetland classification. The quantitative results also demonstrated the capability of jointly using data augmentation, 3D GAN, and Vision Transformer models with overall accuracy, average accuracy, and Kappa index of 75.61%, 73.4%, and 71.87%, respectively, using a disjoint data sampling strategy. Therefore, the proposed deep learning method opens a new window for large-scale remote sensing wetland classification.

Keywords

• Generative adversarial network
• Convolutional neural network
• Wetland classification
• New Brunswick
• Vision Transformer (ViT)
• Deep learning