Facial Expression Recognition From Depth Video With Patterns of Oriented Motion Flow

Abstract

In this paper, we propose a novel feature representation method by a new feature descriptor, named patterns of oriented motion flow (POMF) from the optical flow information, to recognize the proper facial expression from the facial video. The POMF computes different directional motion information and encodes the directional flow information with enhanced local texture micro patterns. As it captures the spatial temporal changes of facial movements through optical flow and enables us to observe both local and global structures, it shows its robustness in recognizing facial information. Finally, the POMF histogram is used to train the expression model through the hidden Markov model (HMM). To train through the HMM, the objective sequences are produced by the generation of a codebook using the K-means clustering technique. The performance of the proposed method has been evaluated over the RGB and depth camera-based video. Experimental results demonstrate that the proposed POMF descriptor is more robust in extracting facial information and provides a higher classification rate than other existing promising methods.

Keywords

• Depth image
• facial expression recognition
• local binary pattern
• optical flow
• patterns of oriented motion flow (POMF)