SSTNet: A Network Based on Modeling of Semantic-Spatiotemporal Information for Semantic Change Detection of Remote Sensing Images

Abstract

Semantic change detection (SCD) aiming to localize changes and identify their types is a complex pixel-level classification task. In SCD, detecting changes of narrow and long objects (NLOs) common in various scenarios is more challenging than detecting these of typical small objects, primarily because NLOs' narrowness makes them carry local details, while their length gives them a global view. The fusion of semantic with spatial information and the interaction of semantic with spatiotemporal information are crucial to accurately detecting changed NLOs. Insufficient utilization of these pieces of information carried by multilayer semantic feature and change feature would limit the representation of the characteristics of changed NLOs, thereby leading to missed detection and false detection. We propose a new model that integrates semantic, spatial, and temporal information, named SSTNet, for SCD. This model utilizes a designed multilayer feature fusion module to fuse features from different layers in the encoder, leveraging global semantic information and local spatial information. In addition, the semantic change feature interaction module facilitates the interaction between semantic feature and change feature, enhancing the correlation between semantic information and spatial information. A series of experiments conducted on two public datasets demonstrate that SSTNet improves the detection accuracy of changed NLOs and outperforms state-of-the-art methods.

Keywords

• Change detection
• narrow and long objects (NLOs)
• remote sensing images
• semantic change detection (SCD)