Human-Based Interaction Analysis via Automated Key Point Detection and Neural Network Model

Abstract

The human interaction with an object is one of the most challenging domains in real-life applications, such as smart homes, surveillance, medical, education, safety-based application of computer vision, and artificial intelligence. In this research article, we have proposed a framework for human and object interaction in real-life examples such as sports and other activities. Initially, we reviewed video-based data by considering the three state-of-the-art data sets. Preprocessing steps have been followed to avoid extra costs, such as video-to-frame conversion, noise reduction and background subtraction. Human silhouette extraction has been performed via the Gaussian mixture model (GMM) and supper pixel model. Next, human body points and object location detection were performed. Finally, human and object-based features have been extracted. To minimize the features replication and to achieve optimized results, we have applied stochastic gradient descent and Restricted Boltzmann Machine; As a result, we have achieved an accuracy of 88.46%, 82.00%, and 88.30% on human body parts recognition over the MPII dataset, UCF_aerial dataset, and wild Dataset respectively. The classification accuracy for the MPII dataset is 92.71%, for the UCF_aerial dataset is 90.60%, and for sports video in the wild Dataset is 92.42%. We have achieved a high accuracy rate compared to other state-of-the-art methods and frameworks due to the complex feature extraction and optimization approach.

Keywords

• Features optimization
• human-object interaction analysis
• human body key points detection
• restricted Boltzmann machine
• stochastic gradient decent
• skeletonization