FT Xtraction: Feature extraction and visualization of conversational video data for social and emotional analysis

Abstract

Conversational video data are widely used to analyze social interaction patterns in various fields, such as social-emotional learning. However, current video processing technologies for social and emotional analysis lack unified feature extraction from video data and video overlays for feature visualization. This paper introduces FT Xtraction to address such limitations. For unified feature extraction, FT Xtraction extracts key base features, such as facial emotion, facial landmarks, and pose keypoints, which are then used to extract derived features such as emotion entropy/synchronicity, pose synchronicity, and physical activeness. FT Xtraction supports video overlay-based feature visualization that allows researchers to examine video data along with extracted features.

Keywords

• Human behavior analysis
• Feature extraction from video data