Three‐dimensional shape reconstruction of objects from a single depth view using deep U‐Net convolutional neural network with bottle‐neck skip connections

Abstract

Abstract Three‐dimensional (3D) shape reconstruction of objects requires multiple scans and complex reconstruction algorithms. An alternative approach is to infer the 3D shape of an object from a single depth image (i.e. single depth view). This study presents such a 3D shape reconstructor based on U‐Net 3D‐convolutional neural network (3D‐CNN) with bottle‐neck skipped connection blocks (U‐Net BNSC 3D‐CNN) to infer the 3D shapes of objects from only a single depth view. The BNSC block is a fully convolutional block that uses skip connections to improve the performance of the sequential 3D‐convolutional layers of U‐Net. The primary advantage of U‐Net BNSC 3D‐CNN is improving the accuracy of shape reconstruction while reducing the computational load. The evaluation of the proposed U‐Net BNSC 3D‐CNN uses unseen views from trained and untrained objects with two public databases, ShapeNet and Grasp database. Our reconstructor achieves 72.17% and 69.97% accuracy in terms of the Jaccard similarity index for trained and untrained objects, respectively, with the ShapeNet database, whereas previous reconstructor based on 3D‐CNN achieves 66.43% and 58.35%. With Grasp database, our reconstructor achieves 87.03% and 85.35%, whereas 3D‐CNN 76.52% and 76.02%. Also, our U‐Net BNSC 3D‐CNN reduces the computational load of the standard 3D‐CNN reconstructor by 6.67% in the computation time and by 98.69% in the number of trainable parameters.

Keywords

• convolution
• feature extraction
• image classification
• image recognition
• image reconstruction
• object recognition