Training Compact Change Detection Network for Remote Sensing Imagery

Abstract

Change Detection (CD) is a hot remote sensing topic where the change zones are highlighted by analyzing bi-temporal or multi-temporal images. Recently, Deep learning (DL) paved the road to implement various reliable change detection approaches that overcome traditional CD methods limitation. However, high performance DL based approaches have explosion number of parameters that demanded extensive computation and memory usage in addition to large volumes of training data. To address this issue, we proposed a teacher-student setting for remote sensing imagery change detection. To distill the knowledge from the over-parameterized Siamese teacher network, we proposed tiny student network that was trained using the obtained categorical distribution of probability from the teacher paired Softmax output at high temperature. Practical Swarm Optimization (PSO) was applied in order to optimally configure student architecture. Finally, ample experiments were conducted on LEVIR-CD dataset. Also, we introduced EGSAR-CD dataset, which contains of a large set of bi-temporal SAR images with 460 image pairs ( $256 \times 256$ ). Experiment results indicate that we can reach up to $5.4\times $ reduction rate in number of parameters with loss of accuracy between 5% and 6% on the LEVIR-CD and EGSAR-CD datasets utilizing self-knowledge distillation.

Keywords

• Change detection (CD)
• deep learning (DL)
• knowledge distillation (KD)
• Siamese network
• teacher-student setting
• practical swarm optimization (PSO)