A new method for reconstructing building model using machine learning

Abstract

Three-dimensional (3D) model reconstruction is used in an increasing number of fields related to construction, such as urban planning, mobile communication planning, and solar power assessment. Existing 3D reconstruction models mostly focus on precise measurements, such as laser scanning and ultrasonic mapping. Although these methods can achieve very precise results, they require specific equipment, which is typically expensive. The essence of 3D reconstruction is to infer the overall view of a building through pictures from previously taken perspectives, thereby obtaining pictures from unfamiliar perspectives. In this study, the rendering method is adopted as the starting point, and architectural features are learned by training a neural network to provide the necessary information for rendering. Unlike the more popular projection-based raster rendering method, this study uses a point-based volume rendering method and light sampling to detect architectural features. This rendering method requires the color and density of specific sampling points. Therefore, this study attempts to train a neural network to fit a five-dimensional (5D) function. The input to this function is a 5D vector, including the position (x, y, z) and viewing direction (θ, φ), and the output is the color and density of this point when viewed from this direction. This study adopts the positional encoding method, which reduces the scale of the network and increases both the training and rendering speeds. Our method can train a usable network in dozens of seconds and render a building at 30–60 frames per second.

Keywords

• building 3d reconstruction
• volume rendering
• machine learning
• nerf
• instant-ngp