Semantic-Enhanced Foundation Model for Coastal Land Use Recognition from Optical Satellite Images

Abstract

Coastal land use represents the combination of various land cover forms in a coastal area, which helps us understand the historical events, current conditions, and future progress of a coastal area. Currently, the emergence of high-resolution optical satellite images significantly extends the scope of coastal land cover recognition, and deep learning models provide a significant possibility of extracting high-level abstract features from an optical satellite image to characterize complicated coastal land covers. However, recognition systems for labeling are always defined differently for specific departments, organizations, and institutes. Moreover, considering the complexity of coastal land uses, it is impossible to create a benchmark dataset that fully covers all types of coastal land uses. To improve the transferability of high-level features generated by deep learning to reduce the burden of creating a massive amount of labeled data, this paper proposes an integrated framework to support semantically enriched coastal land use recognition, including foundation model-powered multi-label coastal land cover classification and conversion from coastal land cover mapping into coastal land use semantics with a vector space model (VSM). The experimental results prove that the proposed method outperformed the state-of-the-art deep learning approaches in complex coastal land use recognition.

Keywords

• transfer learning
• deep learning
• coastal land use recognition
• optical satellite image
• foundation models