Change detection and update of 3D sparse map by merging geometry and appearance

Abstract

Large-scale 3D sparse maps generated by Structure-from-Motion (SfM) from images play an important role in many applications, including visual localization, augmented reality, etc. In these scenarios, the timeliness of the map i.e., detecting changes in the map and performing partial updates, is crucial. To solve this problem, in this paper we propose a novel method for 3D sparse map change detection and updating, to maintain the SfM map continuously over time. The core idea of this paper is to simultaneously detect the appearance and geometry changes of 3D map points, so as to find regions with significant changes, and update these regions locally without changing most of the stable map regions. In the proposed method, a local 3D map containing changing areas is computed from newly captured images by SfM and aligned to the old map according to the locations of new images in the new local map and their registration in the old map. Next, the overlapping map is partitioned into regular grids and the appearance uncertainty and geometry uncertainty are measured on each grid cell individually. Then the grid cells are labeled as changed or unchanged using Markov Random Field optimization by taking both cell uncertainty and consistency of adjacent cells into consideration. Finally, the visible old images of the point cloud in the changed cells are replaced with the corresponding visible new images, and the old map is updated by a local Bundle Adjustment. Experimental results on 3D maps reconstructed by aerial and ground images demonstrate the effectiveness and robustness of the proposed method.

Keywords

• Change detection
• 3D map update
• 3D sparse map