Glass façade segmentation and repair for aerial photogrammetric 3D building models with multiple constraints

Abstract

Aerial photogrammetric three-dimensional (3D) city models provide photorealistic virtual scenes for various applications, such as urban planning, autonomous driving, emergency management, etc. With the rapid progress of urbanization, glass façades commonly exist on buildings, which cannot be well reconstructed with geometric deformation and distortion due to the limitation of the image-based Multi-View Stereo (MVS) modeling pipeline. The distorted glass façades make building models visually unreal and messy. This paper proposes a workflow to segment and repair glass façades in aerial photogrammetric 3D building models with multiple constraints to resolve the problem, mainly including two aspects: 1) projection-based glass façade segmentation with geometric information and multi-view constraints, and 2) glass façade model repair via mesh smoothing and texture mapping. First, an efficient approach for façade information extraction is proposed, which utilizes the density of the reprojected point clouds and elevation differences because the typical structure of building façades is usually vertically aligned to the ground surface. Second, we project the façade mesh onto the multi-view oblique aerial images. The glass façades are predicted by a deep learning-based semantic segmentation method to distinguish the glass façades. Finally, based on the extracted glass façades, we remedy the geometric deformation using a mesh denoising method by grouping strategy with edge constraints. After smoothing the mesh structure of glass façades, we remap the texture to generate the completed 3D models with optimized glass façades. Our method leverages multiple constraints, including multi-view constraints, façade structure constraints, grouping strategy, and boundary constraints. The multi-view and façade structure constraints help decrease the adverse effects caused by false segmentation; the grouping strategy and boundary constraints can avoid disorders in some specific areas and preserve the sharp feature in the adjacent regions. Thus, our approach is robust and does not worsen the 3D model. Experiments demonstrate that our workflow is feasible for glass façade segmentation and repair for aerial photogrammetric 3D building models.

Keywords

• Glass façades
• Aerial photogrammetric 3D modeling
• Multi-view oblique aerial image
• Segmentation
• 3D building model repair
• Multiple constraints