Research on 3D Reconstruction of Furniture Based on Differentiable Renderer

Abstract

Due to the self-obscuration, traditional 3D reconstruction algorithms have difficulty in recovering the 3D structure of an object from a single image. With the rapid development of convolutional neural networks, 3D reconstruction based on deep learning has attracted a wide range of attention from researchers. However, it is expensive to obtain the 3D supervised data corresponding to the objects. To solve the above problems, we combine convolutional neural networks with differentiable renderer and propose the Mesh_CA in this paper, which enables reconstruction of a single image without 3D supervision data. Specifically, an ellipsoid is first initialized for each input single view, and then the features extracted by the convolutional neural network are used to guide the deformation of the ellipsoid to obtain the generated 3D object; After that, the generated object is passed into the differentiable renderer and its corresponding contour information is output; finally, calculating the error between the predicted contour and the real one, and the final 3D object is obtained after training and testing. By training and testing on five types of furniture objects on a large-scale public dataset ShapeNet, the performance of the proposed Mesh_CA surpasses current classical methods.

Keywords

• Deep learning
• 3D reconstruction
• differentiable renderer
• attention mechanism