Permutohedral Refined UNet: Bilateral Feature-Scalable Segmentation Network for Edge-Precise Cloud and Shadow Detection

Abstract

Remote sensing images are usually contaminated by opaque cloud and shadow regions when acquired, and thus cloud and shadow detections become one of the essential prerequisites for the restoration of the objects of interest before further processing and analysis. Such a detection issue can be considered an image segmentation task supported by cutting-edge machine learning techniques, but edge-precise performance is still challenging. In this article, we therefore present Permutohedral Refined UNet, a two-stage, spectral feature-scalable pipeline to take into account the edge-precise segmentation, relatively feasible efficiency, global refinement, and spectral feature scalability. Specifically, given local tiles of a full-resolution image, the local unary potential is created in terms of the logits yielded by the UNet backbone with pretrained parameters, and global refinement is then performed by a following inference of our custom conditional random field (CRF); this pipeline can finally yield edge-refined results for cloud and shadow segmentation. In particular, a relatively efficient implementation is also given using the Eigen library, making it possible to run such inference in a practically time-saving way; our implementation can also create bilateral kernels with multi-spectral features, giving rise to a relatively significant improvement in shadow retrieval in comparison to the CRF built only with RGB bilateral features. Extensive experiments report that our implementation can achieve edge-refined cloud and shadow segmentation in a relatively efficient, globally refined, and spectral feature-scalable way, in terms of the practical performance on both the Landsat 8 OLI and the RICE datasets.

Keywords

• Bilateral feature scalability
• cloud and shadow detection
• object edge refinement
• permutohedral lattice-based inference of conditional random fields (CRFs)