Automatic corrections of human body depth maps using deep neural networks

Abstract

This paper presents an approach to correcting misclassified pixels in depth maps representing parts of the human body. A misclassified pixel is a pixel of a depth map which, incorrectly, has the ‘background’ value and does not accurately reflect the distance from the sensor to the body being scanned. A completely automatic, deep learning based solution for depth map correction is proposed. As an input, the solution requires a color image and a corresponding erroneous depth map. The input color image is segmented using deep neural network for human body segmentation. The extracted segments are further used as guidance to find and amend the misclassified pixels on the depth map using a simple average based filter. Unlike other depth map refinement solutions, this paper designs a method for the improvement of the input depth map in terms of completeness instead of precision. The proposed method does not exclude the application of other refinement methods. Instead, it can be used as the first step in a depth map enhancement pipeline to determine approximate depths for erroneous pixels, while other refinement methods can be applied in a second step to improve the accuracy of the recovered depths.

Keywords

• depth map refinement
• human body
• photogrammetry
• deep learning
• segmentation