Personalised face neutralisation based on subspace bilinear regression

Abstract

Expression face neutralisation helps to improve the performance of expressive face recognition with one single neutral sample in gallery per subject. For learning‐based expression neutralisation, the virtual neutral face totally relies on training samples, which removes person‐specific characters from the neutralised face. Bilinear kernel rank reduced regression (BKRRR) algorithm is designed in a virtual subspace to simultaneously and efficiently generate both virtual expressive and neutral images from training samples. An expression mask is then established using grey and gradient differences of the two images. The test expression image is transformed to neutral template by piece‐wise affine warp (PAW). Using the virtual BKRRR neutral image as source, the PAW image as destination and the area covered by expression mask as clone area, an image fusion strategy based on Poisson equation is then designed, which achieves virtual neutralised face image with person‐specific characters preserved. From experiments on the CMU Multi‐PIE databases, it could be observed that the neutral faces synthesised by the proposed method could effectively approximate the real ground truth expressive faces, and greatly improve the performance of classic face recognition algorithms on expression variant problems.

Keywords

• personalised face neutralisation
• subspace bilinear regression
• expressive face recognition
• learning-based expression neutralisation
• virtual neutral face
• bilinear kernel rank reduced regression