A method for reconstructing 3D scenes based on the use of convolutional neural networks, filtering by distance and using an “octree”

Abstract

Background. The object of research is machine vision systems. The subject of research is the methods of reconstruction of three-dimensional scenes. The aim of the work is to develop a method for reconstruction of three-dimensional scenes, including the stages of identification of three-dimensional objects, segmentation (selection) and filtering of their constituent points in the original cloud. Materials and methods. The Block-Matching Algorithm method used to generate a depth map, the Mask R-CNN convolutional neural network – to identify and segment objects in the external environment, the OctTree method – to filter the point cloud, the Delaunay triangulation method - to generate a threedimensional model. Results. Based on the proposed method for reconstructing threedimensional scenes, a software product was developed in the python programming languages (TensorFlow virtual environment for implementing the Mask RCNN convolutional network) and C #, which implements the formation of a three-dimensional model of the road surface. Conclusions. The resulting three-dimensional model of the road surface can then be used to determine the following parameters: boundaries, axis of the road surface, geometric dimensions of defects (potholes, waves, depressions, chipping, sweating, protrusions, cracks), longitudinal evenness index (IRI).

Keywords

• three-dimensional scene
• reconstruction
• segmentation
• filtering
• recognition
• convolutional neural network