A novel motion key frame extraction and video stream classification based on reinforcement learning and feature fusion

Abstract

In order to solve the problem of missing detection and false detection caused by the inaccuracy of motion feature extraction in the existing video key frame extraction algorithms, a reinforcement learning and feature fusion for key frame extraction algorithm and video stream classification is proposed. The fusion features are obtained by combining the original statistical features via S-GCN and ResNet50 network. Some fusion features are more effective than the original statistical features. Therefore, in order to extract useful information for classification, the original features and fusion features are combined to produce composite features. At the same time, the number of features increases and there are redundant and irrelevant features. Embedded feature selection method and random forest classifier are used to select the best feature subset. Finally, the attention mechanism is used to calculate the importance of video frames, and reinforcement learning is used to extract and optimize key frames. The experimental results show that the new algorithm can solve the problem of error detection in the key frame extraction of motion video, and performs well in the detection of video frames containing key actions. The algorithm has high accuracy and strong stability.

Keywords

• key frame extraction
• reinforcement learning
• feature fusion
• video stream classification
• s-gcn
• resnet50