IEEE Access (Jan 2023)
3DHR-Co: A Collaborative Test-Time Refinement Framework for In-the-Wild 3D Human-Body Reconstruction Task
Abstract
The task of 3D human-body reconstruction (3DHR), which mostly utilizes parametric pose and shape representations, has witnessed significant advances in recent years. However, the application of 3DHR techniques in handling real-world in-the-wild data, still faces limitations. Training the 3DHR model in such scenario with 3D human pose’s ground truth (GT) is non-trivial. Curating the accurate 3D human pose GT for in-the-wild scenes remains difficult due to various factors. Recent test-time refinement approaches on 3DHR task leverage 2D off-the-shelf human keypoints information to support the lack of 3D supervision on in-the-wild data. However, we observed that additional 2D supervision alone could cause overfitting issue on common 3DHR backbones, making the 3DHR test-time refinement task seem intractable. We answer this challenge by proposing a strategy that complements 3DHR test-time refinement work under a collaborative approach. Specifically, we initially apply a pre-adaptation approach that works by collaborating various 3DHR models in a single framework to directly improve their initial outputs. This approach is then further combined with the test-time adaptation work under specific settings that minimize the overfitting issue to further boost the 3DHR performance. The whole framework is termed as 3DHR-Co, and on the experiment side, the proposed work can significantly enhance the scores of common classic 3DHR backbones up to -34 mm pose error suppression, putting them among the top list on the in-the-wild benchmark data. Such achievement shows that our approach helps unveil the true potential of the common classic 3DHR backbones. Based on these findings, we further investigate various settings on the proposed framework to better elaborate the capability of our collaborative strategy in the 3DHR task.
Keywords
