Is the ROC curve a reliable tool to compare the validity of landslide susceptibility maps?

Abstract

This study investigates how effective the receiver operating characteristic (ROC) curve is for comparing the reliability of landslide susceptibility maps (LSMs). In this regard, we selected a basin prone to landslides in northern Iran and employed frequency ratio and weight-of-evidence methods for modelling. Modelling for each method was done by considering three fractions of training/test landslides (50/50, 60/40, and 70/30) and, in addition, three random seeds for each of these fractions, leading to produce 18 LSMs. Validation rates of LSMs were obtained through calculation of the area under the ROC curve. Moreover, Cohen's kappa index was calculated to reveal the magnitude of the agreement between the maps. Results showed that all LSMs, despite having equal validation rates, were considerably different in terms of spatial prediction pattern. It is concluded that, although as a prevalent validation tool, ROC curve is only an indicator of the general reliability of geographical prediction maps and cannot reveal the uncertainty of spatial prediction patterns. Therefore, to reduce the conflicts between the maps and create a single reliable map, all LSMs were merged using the overlaying statistical functions that can extract the best possible zone pattern from all the overlapped patterns.

Keywords

• Model uncertainty
• landslide susceptibility map
• ROC method
• Cohen's kappa index
• spatial prediction map