STMNet: Scene Classification-Assisted and Texture Feature-Enhanced Multiscale Network for Large-Scale Urban Informal Settlement Extraction From Remote Sensing Images

Abstract

Automatic urban informal settlement (UIS) extraction based on high-resolution remote sensing image (HRI) is of great significance for urban planning and management. This study proposes a scene classification-assisted and texture feature-enhanced multiscale network (STMNet) for UIS extraction. First, STMNet takes HRI and computed handcrafted texture feature (HTF) as input. Second, it employs a pseudo-siamese network to extract multidimensional deep features from HRI and HTF, respectively. In addition, a feature attention fusion module is constructed to fuse the aforementioned features. Finally, skip connection and feature decoder are utilized to obtain UIS extraction results. In detail, considering the sparse and dispersed distribution of UIS, a scene information aggregation and classification module is constructed to determine whether the input image patch contains UIS. For the characteristics of high spatial heterogeneity and various shapes and scales of UIS, an improved atrous spatial pyramid pooling is presented to extract multiscale and multireceptive field features. An edge loss function is applied during network training to minimize errors in the edge regions of UIS. The effectiveness of STMNet is tested on a self-produced UIS extraction dataset and the publicly available UIS-Shenzhen dataset. Quantitative results demonstrate that STMNet achieved the best performance in terms of $\text{fa}$, $F1$, and $\text{IoU}$. The $\text{mr}$ is slightly higher than that of MAResU-Net and UisNet. In addition, STMNet achieved the best visual interpretation results and the fastest inference speed on the self-produced UIS extraction dataset.

Keywords

• Handcrafted texture feature (HTF)
• high-resolution remote sensing image (HRI)
• scene classification
• semantic segmentation
• urban informal settlement (UIS)