Skeleton‐aware implicit function for single‐view human reconstruction

Abstract

Abstract The aim is to reconstruct a complete and detailed clothed human from a single‐view input. Implicit function is suitable for this task because it represents fine shape details and varied topology. Current methods, however, often suffer from artefacts such as broken or disembodied body parts, missing details, or depth ambiguity due to the ambiguity and complexity of human articulation. The main issue observed by the authors is structure‐agnostic. To address these problems, the authors fully utilise the skinned multi‐person linear (SMPL) model and propose a method using the Skeleton‐aware Implicit Function (SIF). To alleviate the broken or disembodied body parts, the proposed skeleton‐aware structure prior makes the skeleton awareness into an implicit function, which consists of a bone‐guided sampling strategy and a skeleton‐relative encoding strategy. To deal with the missing details and depth ambiguity problems, the authors’ body‐guided pixel‐aligned feature exploits the SMPL to enhance 2D normal and depth semantic features, and the proposed feature aggregation uses the extra geometry‐aware prior to enabling a more plausible merging with less noisy geometry. Additionally, SIF is also adapted to the RGB‐D input, and experimental results show that SIF outperforms the state‐of‐the‐arts methods on challenging datasets from Twindom and Thuman3.0.

Keywords

• 3D human reconstruction
• deep learning
• neural network