A feature matching and compensation method based on importance weighting for occluded human pose estimation

Abstract

Human pose estimation is a hot research in the field of computer vision. Existing algorithms perform well in normal scenarios but struggle in occlusion environments. In order to address occlusion problems, we firstly design a multi-scale feature reinforcement backbone (MFRI) to obtain accurate features, which contains three sub-modules. The first sub-module reinforces detailed keypoint features and eliminates redundant background, The second sub-module assigns different weights to different scale information based on importance. The fourth sub-module integrates different scale feature information using a cascaded method. In addition, we design an occlusion handling method. To begin with, we design a multi-scale feature extractor (MSFE) to construct human topological feature, and propose a feature similarity matching method between keypoint detail features and topological features to identify occluded keypoints and eliminate obstacle features. Based on similarity, we design a feature compensation method to extract common attributes among all keypoints and unique features of each keypoint from multiple aspects to compensate for occluded features. Finally, we propose an adjacency matrix improvement method to enhance its ability for describing node relationships. We conduct comparative experiments on the COCO2017 dataset, COCO-Wholebody dataset, and CrowdPose dataset, achieving accuracy of 78.8%, 66.3%, and 77.8%, respectively. Additionally, we design a series of ablation experiments and visualizations. The results show that our method has better performance in handling occlusions.

Keywords

• Occluded human pose estimation
• Importance weighting
• Features matching
• Features compensation
• Improved adjacency matrix