STA-AgriNet: A Spatio-Temporal Attention Framework for Crop Type Mapping from Fused Multi-Sensor Multi-Temporal SITS

Abstract

Precise and timely crop type mapping delivers insights into crop growth statistics and ensures food security for growing economies. Automated mapping is crucial in several agricultural applications, including crop wear assessment and yield forecasting. The high-resolution multispectral optical data can deliver essential spatial-spectral characteristics; however, these are typically impeded by unfavorable weather and obstructions, resulting in poor classification. Recent advancements in multisensor data fusion techniques have focused on integrating optical data with auxiliary synthetic aperture radar (SAR) data to mitigate misclassification issues. However, current optical-SAR fusion techniques have yet to effectively address the incorporation of spatial-spectral characteristics with long-term temporal dependencies of satellite image time series (SITS). This article proposes an optical-SAR deep fusion framework, STA-AgriNet, that integrates a U-Net encoder–decoder with spatial-temporal attention frameworks to enable superior extraction of long-term spatial-temporal dependencies for reliable crop-type mapping. The spectral spatial feature mapper (SSFM), mixed parallel spatial attention (MPSA), and spatio-temporal attention mapper (STAM) modules of the STA-AgriNet extract key classification-defining, discriminative patterns for semantic segmentation. The STA-AgriNet framework is evaluated against current state-of-the-art (SOTA) methods and demonstrates superior performance across multiple metrics, achieving an accuracy of 83.61% with a minimal inference time of 10.09 s and a compact parameter count of 0.97 million. The model also excels in other key evaluation metrics, establishing its overall effectiveness and efficiency compared to existing techniques.

Keywords

• Crop mapping
• data fusion
• Sentinel-1
• Sentinel-2
• spatial attention
• temporal attention