Approach to 3D face reconstruction through local deep feature alignment

Abstract

Here, the authors propose an end‐to‐end method based on deep learning to reconstruct three‐dimensional (3D) face models from given face images. In the training stage, the authors propose to extract the feature representations from the 3D sample faces and corresponding 2D sample images through the proposed local deep feature alignment (LDFA) algorithm, and estimate an explicit mapping from the 2D features to their 3D counterparts for each local neighbourhood, then the authors learn a feed‐forward deep neural network for each neighbourhood whose parameters are initialised with the parameters obtained in the locality‐aware learning process and the explicit mapping. In the testing stage, the authors only need to feed a given face image to the deep neural network corresponding to the nearest sample image and receive the outputted 3D face model. Extensive experiments have been conducted on both non‐face and face data sets. The authors find that the LDFA algorithm performs better than several popular unsupervised feature extraction algorithms, and the 3D reconstruction results obtained by the proposed method also outperform the comparison methods.

Keywords

• feed-forward deep neural network
• locality-aware learning process
• explicit mapping
• testing stage
• nearest sample image
• outputted 3D face model