Zero-Shot Remote Sensing Scene Classification Based on Automatic Knowledge Graph and Dual-Branch Semantic Correlation Supervision

Abstract

Remote sensing scene knowledge graphs symbolically describe the concepts of scenes and reveal their interrelations, highlighting robust knowledge modeling and inference capabilities in zero-shot remote sensing scene classification tasks. However, current graphs rely heavily on expert manual interpretation, making them susceptible to human biases and difficult to expand. They also lack quantitative description of the degree of spatial relationship associations and fail to adequately represent semantic information. Additionally, they often overlook discriminative local landscapes, which is crucial for accurate scene classification. To address these limitations, this article proposes the “zero-shot remote sensing scene classification via automatic knowledge graph and dual-branch semantic correlation supervision (AKG-DBSS).” This method starts with the scenes and automates the construction of a knowledge graph, “scene-landscape-ground objects” (ASLG-KG), by analyzing the composition and spatial distribution of ground objects in local regions. On this basis, the dual-branch semantic correlation supervised zero-shot remote sensing scene classification network (DBSS) supervises the mapping of semantic features to the visual space via both global and local branches, ensuring the visual space reflects the correlation structure of the semantic space. Extensive experiments on the UCM, AID, and NWPU datasets demonstrate that AKG-DBSS achieves class average accuracy and overall accuracy of up to 98%, and 59.56%, respectively, for the classification of unseen class scenes, with standard deviations below 6.91%, significantly outperforming four other advanced comparative methods. Furthermore, additional experiments prove that ASLG-KG and DBSS are feasible, necessary, and effective, with an accuracy improvement in overall accuracy of over 8.98%.

Keywords

• Automatic “scene-landscape-ground objects” knowledge graph
• dual-branch semantic correlation supervision (DBSS)
• remote sensing scene classification
• zero-shot learning