Extraction buildings from very high-resolution images with asymmetric siamese multitask networks and adversarial edge learning

Abstract

Building extraction from very high-resolution remote-sensing images still faces two main issues: (1) small buildings are severely omitted and the extracted building shapes have a low consistency with ground truths. (2) supervised deep-learning methods have poor performance in few-shot scenarios, limiting the practical application of these methods. To address the first issue, we propose an asymmetric Siamese multitask network integrating adversarial edge learning called ASMBR-Net for building extraction. It contains an efficient asymmetric Siamese feature extractor comprising pre-trained backbones of convolutional neural networks and Transformers under pre-training and fine-tuning paradigms. This extractor balances the local and global feature representation and reduces training costs. Adversarial edge-learning technology automatically integrates edge constraints and strengthens the modeling ability of small and complex building-shaped patterns. Aiming to overcome the second issue, we introduce a self-training framework and design an instance transfer strategy to generate reliable pseudo-samples. We examined the proposed method on the WHU and Massachusetts (MA) datasets and a self-constructed Dongying (DY) dataset, comparing it with state-of-the-art methods. The experimental results show that our method achieves the highest F1-score of 96.06%, 86.90%, and 84.98% on the WHU, MA, and DY datasets, respectively. Ablation experiments further verify the effectiveness of the proposed method. The code is available at: https://github.com/liuxuanguang/ASMBR-Net

Keywords

• Building extraction
• VHR remote-sensing image
• Multitask learning
• Adversarial learning
• Self-learning