A new attention-based deep metric model for crop type mapping in complex agricultural landscapes using multisource remote sensing data

Abstract

Accurate crop mapping is critical for agricultural decisions and food security. Despite the widespread use of machine learning and deep learning in remote sensing for crop classification, mapping crops in mountainous smallholder farming systems remains challenging. In particular, cloudy and rainy weather limits high-quality satellite imagery, potentially limiting the availability of reliable data for classification. Additionally, the substantial intraclass variability among multiple crops further impedes classification accuracy. In this context, this study sought to resolve these two issues by applying a hybrid approach that combines multisource remote sensing data and deep metric learning. For the first challenge, multisource remote sensing data, including Landsat-8, Sentinel-2 and Sentinel-1 data from the Google Earth Engine, were integrated to provide more comprehensive information on crop growth and differences. To address the second challenge, we proposed a 2D-CNN network enhanced by CBAM attention and an online hard example mining strategy. The network focuses on the channel-spatial information of crop samples and their surrounding pixels while promoting the convergence of similar crop samples within the latent feature space and enhancing the separation among different samples. This process is exemplified through a case study of crop mapping in Jiangjin District, Chongqing city, an area representing the typical mountain smallholder farming systems in Southwest China. Compared to six state-of-the-art methods, RF, SVM, XGBoost, ResNet18, and DMLOHM, our approach achieves the highest performance, with 93.99% overall accuracy, a kappa coefficient of 0.9253, and excellent F1 scores across numerous crop categories. The results of this study provide an effective solution for crop classification in complex mountainous regions and have promising potential for mapping under challenging natural conditions.

Keywords

• Crop classification
• Deep metric learning
• Convolutional neural network
• Multisource remote sensing