Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics
Meiqi Zhuang,
Lang Yin,
Youhua Wang,
Yunzhao Bai,
Jian Zhan,
Chao Hou,
Liting Yin,
Zhangyu Xu,
Xiaohui Tan,
YongAn Huang
Affiliations
Meiqi Zhuang
Information Engineering College, Capital Normal University, Beijing 100048, China
Lang Yin
State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China; Flexible Electronics Research Center, Huazhong University of Science and Technology, Wuhan 430074, China
Youhua Wang
State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China; Flexible Electronics Research Center, Huazhong University of Science and Technology, Wuhan 430074, China
Yunzhao Bai
State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China; Flexible Electronics Research Center, Huazhong University of Science and Technology, Wuhan 430074, China
Jian Zhan
State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China; Flexible Electronics Research Center, Huazhong University of Science and Technology, Wuhan 430074, China
Chao Hou
State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China; Flexible Electronics Research Center, Huazhong University of Science and Technology, Wuhan 430074, China
Liting Yin
State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China; Flexible Electronics Research Center, Huazhong University of Science and Technology, Wuhan 430074, China
Zhangyu Xu
State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China; Flexible Electronics Research Center, Huazhong University of Science and Technology, Wuhan 430074, China
Xiaohui Tan
Information Engineering College, Capital Normal University, Beijing 100048, China
YongAn Huang
State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China; Flexible Electronics Research Center, Huazhong University of Science and Technology, Wuhan 430074, China
The facial expressions are a mirror of the elusive emotion hidden in the mind, and thus, capturing expressions is a crucial way of merging the inward world and virtual world. However, typical facial expression recognition (FER) systems are restricted by environments where faces must be clearly seen for computer vision, or rigid devices that are not suitable for the time-dynamic, curvilinear faces. Here, we present a robust, highly wearable FER system that is based on deep-learning-assisted, soft epidermal electronics. The epidermal electronics that can fully conform on faces enable high-fidelity biosignal acquisition without hindering spontaneous facial expressions, releasing the constraint of movement, space, and light. The deep learning method can significantly enhance the recognition accuracy of facial expression types and intensities based on a small sample. The proposed wearable FER system is superior for wide applicability and high accuracy. The FER system is suitable for the individual and shows essential robustness to different light, occlusion, and various face poses. It is totally different from but complementary to the computer vision technology that is merely suitable for simultaneous FER of multiple individuals in a specific place. This wearable FER system is successfully applied to human-avatar emotion interaction and verbal communication disambiguation in a real-life environment, enabling promising human-computer interaction applications.
