Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks

By Zitong Yu, Xiaobai Li, Guoying Zhao. British Machine Vision Conference, 2019

419 citations recorded by the source corpus.

Recent studies demonstrated that the average heart rate (HR) can be measured from facial videos based on non-contact remote photoplethysmography (rPPG). However for many medical applications (e.g., atrial fibrillation (AF) detection) knowing only the average HR is not sufficient, and measuring precise rPPG signals from face for heart rate variability (HRV) analysis is needed. Here we propose an rPPG measurement method, which is the first work to use deep spatio-temporal networks for reconstructing precise rPPG signals from raw facial videos. With the constraint of trend-consistency with ground truth pulse curves, our method is able to recover rPPG signals with accurate pulse peaks. Comprehensive experiments are conducted on two benchmark datasets, and results demonstrate that our method can achieve superior performance on both HR and HRV levels comparing to the state-of-the-art methods. We also achieve promising results of using reconstructed rPPG signals for AF detection and emotion recognition.

Subfields: Biomedical Engineering, Cardiology and Cardiovascular Medicine

View Paper: https://www.semanticscholar.org/paper/364ec52bba0bcd43c6deb588bf7f1d8269be84b9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

1. Zitong Yu
2. Guoying Zhao
3. Xuesong Niu
4. Xiaobai Li
5. Jingang Shi
6. Shiguang Shan
7. Xilin Chen
8. Hu Han

In Cardiology and Cardiovascular Medicine

1. Zitong Yu
2. Guoying Zhao
3. Xuesong Niu
4. Xiaobai Li
5. Jingang Shi
6. Hu Han
7. Shiguang Shan
8. Xilin Chen

Citations in the corpus

• RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation (2019, methodology): “the input video clips using the visualization method proposed in [39].”
• Face Anti-Spoofing with Human Material Perception (2020, methodology): “Meanwhile, several video-level CNN methods are presented to exploit the dynamic spatio-temporal [9,33,34,35] or rPPG [13,14,12,36,37,38] features for PAD.”
• Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement (2020, methodology): “Efficient Temporal Models: Yu et al. [19] have shown that applying 3D convolutional neural networks (CNNs) significantly improves performance and achieves better accuracy compared to using a combination of 2D CNNs and recurrent neural networks.”
• PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer (2021, positive): “Similarly, the existing end-to-end learning based methods (e.g., PhysNet [64], DeepPhys [10], and AutoHR [62]) predict unreliable HR values with large RMSE compared with non-end-to-end learning approaches (e.g., RhythmNet [44], ST-Attention [46], NAS-HR [39], CVD [45], and Dual-GAN [40]).”