RSI-Net: Two-Stream Deep Neural Network for Remote Sensing Images-Based Semantic Segmentation
Shuang He,
Xia Lu,
Jason Gu,
Haitong Tang,
Qin Yu,
Kaiyue Liu,
Haozhou Ding,
Chunqi Chang,
Nizhuan Wang
Affiliations
Shuang He
Jiangsu Key Laboratory of Marine Bioresources and Environment/Jiangsu Key Laboratory of Marine Biotechnology/Co-Innovation Center of Jiangsu Marine Bio-Industry Technology, Jiangsu Ocean University, Lianyungang, China
Xia Lu
Jiangsu Key Laboratory of Marine Bioresources and Environment/Jiangsu Key Laboratory of Marine Biotechnology/Co-Innovation Center of Jiangsu Marine Bio-Industry Technology, Jiangsu Ocean University, Lianyungang, China
Department of Electrical and Computer Engineering, Dalhousie University, Halifax, NS, Canada
Haitong Tang
Jiangsu Key Laboratory of Marine Bioresources and Environment/Jiangsu Key Laboratory of Marine Biotechnology/Co-Innovation Center of Jiangsu Marine Bio-Industry Technology, Jiangsu Ocean University, Lianyungang, China
Qin Yu
School of Computer Engineering, Jiangsu Ocean University, Lianyungang, China
Kaiyue Liu
Jiangsu Key Laboratory of Marine Bioresources and Environment/Jiangsu Key Laboratory of Marine Biotechnology/Co-Innovation Center of Jiangsu Marine Bio-Industry Technology, Jiangsu Ocean University, Lianyungang, China
Haozhou Ding
School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, China
Jiangsu Key Laboratory of Marine Bioresources and Environment/Jiangsu Key Laboratory of Marine Biotechnology/Co-Innovation Center of Jiangsu Marine Bio-Industry Technology, Jiangsu Ocean University, Lianyungang, China
For semantic segmentation of remote sensing images (RSI), trade-off between representation power and location accuracy is quite important. How to get the trade-off effectively is an open question, where current approaches of utilizing very deep models result in complex models with large memory consumption. In contrast to previous work that utilizes dilated convolutions or deep models, we propose a novel two-stream deep neural network for semantic segmentation of RSI (RSI-Net) to obtain improved performance through modeling and propagating spatial contextual structure effectively and a decoding scheme with image-level and graph-level combination. The first component explicitly models correlations between adjacent land covers and conduct flexible convolution on arbitrarily irregular image regions by using graph convolutional network, while densely connected atrous convolution network (DenseAtrousCNet) with multi-scale atrous convolution can expand the receptive fields and obtain image global information. Extensive experiments are implemented on the Vaihingen, Potsdam and Gaofen RSI datasets, where the comparison results demonstrate the superior performance of RSI-Net in terms of overall accuracy (91.83%, 93.31% and 93.67% on three datasets, respectively), F1 score (90.3%, 91.49% and 89.35% on three datasets, respectively) and kappa coefficient (89.46%, 90.46% and 90.37% on three datasets, respectively) when compared with six state-of-the-art RSI semantic segmentation methods.
