A Transfer Learning Approach for Landslide Semantic Segmentation Based on Visual Foundation Model

Abstract

Landslides are one of the most destructive natural disasters in the world, threatening human life and safety. With excellent performance as a foundation model for image segmentation, the segment anything model (SAM) has provided a novel paradigm for semantic segmentation research. However, the lack of remote sensing images in the SAM training data limits its ability to recognize landslides. In addition, despite the transfer learning approach can transfer SAM feature extraction capability to the landslide segmentation task, but it will consume a lot of computational resources and training time. In order to solve these challenges, this study proposes a TransLandSeg model that transfers the segmentation capability of SAM while learning landslide features at a low training cost. To limit model training parameters, the adaptive transfer learning (ATL) module is purposely designed, the image encoder is frozen during model training, only the ATL module and mask decoder are trained, and the knowledge learned from the ATL module is input into the original network. Moreover, to select the best ATL module, we also designed 9 kinds of ATL modules and analyzed the accuracy of the TransLandSeg model with different ATL modules. We selected the Bijie landslide dataset and the Landslide4Sense dataset for model training and testing. The experiment results show that the TransLandSeg model increases the mean intersection over union by 1.48% –13.01% compared to other state-of-the-art semantic segmentation models. In addition, TransLandSeg requires only 1.3% of SAM parameters to enable SAM's powerful capabilities to transfer to landslide segmentation.

Keywords

• Foundation model
• landslide recognition
• semantic segmentation
• transfer learning