RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation

By Xuesong Niu, Shiguang Shan, Hu Han, Xilin Chen. IEEE Transactions on Image Processing, 2019

398 citations recorded by the source corpus.

Heart rate (HR) is an important physiological signal that reflects the physical and emotional status of a person. Traditional HR measurements usually rely on contact monitors, which may cause inconvenience and discomfort. Recently, some methods have been proposed for remote HR estimation from face videos; however, most of them focus on well-controlled scenarios, their generalization ability into less-constrained scenarios (e.g., with head movement, and bad illumination) are not known. At the same time, lacking large-scale HR databases has limited the use of deep models for remote HR estimation. In this paper, we propose an end-to-end RhythmNet for remote HR estimation from the face. In RyhthmNet, we use a spatial-temporal representation encoding the HR signals from multiple ROI volumes as its input. Then the spatial-temporal representations are fed into a convolutional network for HR estimation. We also take into account the relationship of adjacent HR measurements from a video sequence via Gated Recurrent Unit (GRU) and achieves efficient HR measurement. In addition, we build a large-scale multi-modal HR database (named as VIPL-HR1), which contains 2,378 visible light videos (VIS) and 752 near-infrared (NIR) videos of 107 subjects. Our VIPL-HR database contains various variations such as head movements, illumination variations, and acquisition device changes, replicating a less-constrained scenario for HR estimation. The proposed approach outperforms the state-of-the-art methods on both the public-domain and our VIPL-HR databases.1VIPL-HR is available at: http://vipl.ict.ac.cn/view_database.php?id=15

Subfields: Biomedical Engineering, Cardiology and Cardiovascular Medicine

View Paper: https://www.semanticscholar.org/paper/6ce235f0235d00a9c9d9c29dbcef8aeb63f9ec50

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

1. Jingang Shi
2. Zitong Yu
3. Guoying Zhao
4. Philip H. S. Torr
5. Yuming Shen
6. Hengshuang Zhao
7. Xuesong Niu

In Cardiology and Cardiovascular Medicine

1. Jingang Shi
2. Zitong Yu
3. Guoying Zhao
4. Philip H. S. Torr
5. Yuming Shen
6. Hengshuang Zhao
7. Xuesong Niu

Citations in the corpus

• PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer (2021, positive): “Experiments of rPPG-based physiological measurement for three types of physiological signals, i.e., heart rate (HR), heart rate variability (HRV), and respiration frequency (RF), are conducted on four benchmark datasets (VIPL-HR [44], MAHNOB-HCI [51], MMSE-HR [54], and OBF [28]).”