Mixing body‐parts model for 2D human pose estimation in stereo videos

Abstract

This study targets 2D articulated human pose estimation (i.e. localisation of body limbs) in stereo videos. Although in recent years depth‐based devices (e.g. Microsoft Kinect) have gained popularity, as they perform very well in controlled indoor environments (e.g. living rooms, operating theatres or gyms), they suffer clear problems in outdoor scenarios and, therefore, human pose estimation is still an interesting unsolved problem. The authors propose here a novel approach that is able to localise upper‐body keypoints (i.e. shoulders, elbows, and wrists) in temporal sequences of stereo image pairs. The authors’ method starts by locating and segmenting people in the image pairs by using disparity and appearance information. Then, a set of candidate body poses is computed for each view independently. Finally, temporal and stereo consistency is applied to estimate a final 2D pose. The authors’ validate their model on three challenging datasets: ‘stereo human pose estimation dataset’, ‘poses in the wild’ and ‘INRIA 3DMovie’. The experimental results show that the authors’ model not only establishes new state‐of‐the‐art results on stereo sequences, but also brings improvements in monocular sequences.

Keywords

• stereo videos
• mixing body-parts model
• 2D articulated human pose estimation
• Microsoft Kinect
• controlled indoor environments
• localise upper-body keypoints