A Comprehensive Study on the Robustness of Deep Learning-Based Image Classification and Object Detection in Remote Sensing: Surveying and Benchmarking

Abstract

Deep neural networks (DNNs) have found widespread applications in interpreting remote sensing (RS) imagery. However, it has been demonstrated in previous works that DNNs are susceptible and vulnerable to different types of noises, particularly adversarial noises. Surprisingly, there has been a lack of comprehensive studies on the robustness of RS tasks, prompting us to undertake a thorough survey and benchmark on the robustness of DNNs in RS. This manuscript conducts a comprehensive study of both the natural robustness and adversarial robustness of DNNs in RS tasks. Specifically, we systematically and extensively survey the robustness of DNNs from various perspectives such as noise type, attack domain, and attacker’s knowledge, encompassing typical applications such as object detection and image classification. Building upon this foundation, we further develop a rigorous benchmark for testing the robustness of DNN-based models, which entails the construction of noised datasets, robustness testing, and evaluation. Under the proposed benchmark, we perform a meticulous and systematic examination of the robustness of typical deep learning algorithms in the context of object detection and image classification applications. Through comprehensive survey and benchmark, we uncover insightful and intriguing findings, which shed light on the relationship between adversarial noise crafting and model training, yielding a deeper understanding of the susceptibility and limitations of various DNN-based models, and providing guidance for the development of more resilient and robust models.