Heart Rate Estimation From Remote Photoplethysmography Based on Light-Weight U-Net and Attention Modules

Abstract

Cardiac signals are frequently used in disease and emotion analyses. However, current measurement methods mostly require direct contact. Remote photoplethysmography (rPPG) has been proposed in recent years which measures minute variations in color on the face due to blood volume changes as the heart pumps, using a consumer grade camera. In this study, we proposed a deep learning framework based on a light-weight and task-adapted version of U-Net to extract rPPG. The face video was converted into multiscale spatio-temporal map (MSTmap) as input to the network. Two types of attention mechanisms were added, namely variations of the squeeze-and-excitation block (SE block), which compresses global information to enhance the channel and ROI signals, and the multihead attention block with position encoding, which extracts information from different parts of the signal. We further propose using virtual PPG (vPPG) as a replacement for PPG ground-truth so that the model focuses on learning the peak information instead of morphological details. Extensive experiments were conducted using the UBFC-rPPG dataset for heart rate (HR) and heart rate variability (HRV) estimations. The model achieved a root-mean-square error of 0.78 bpm and correlation coefficient of 0.99 in heart rate estimation, which is comparable to state-of-the-art while being more light-weight.

Keywords

• Attention
• remote photoplethysmography
• remote heart rate estimation
• spatio–temporal map