Remote Heart Rate Measurement From Highly Compressed Facial Videos: An End-to-End Deep Learning Solution With Video Enhancement
By Zitong Yu, Wei Peng, Xiaobai Li, Xiaopeng Hong, Guoying Zhao. IEEE International Conference on Computer Vision, 2019
332 citations recorded by the source corpus.
Remote photoplethysmography (rPPG), which aims at measuring heart activities without any contact, has great potential in many applications (e.g., remote healthcare). Existing rPPG approaches rely on analyzing very fine details of facial videos, which are prone to be affected by video compression. Here we propose a two-stage, end-to-end method using hidden rPPG information enhancement and attention networks, which is the first attempt to counter video compression loss and recover rPPG signals from highly compressed videos. The method includes two parts: 1) a Spatio-Temporal Video Enhancement Network (STVEN) for video enhancement, and 2) an rPPG network (rPPGNet) for rPPG signal recovery. The rPPGNet can work on its own for robust rPPG measurement, and the STVEN network can be added and jointly trained to further boost the performance especially on highly compressed videos. Comprehensive experiments are performed on two benchmark datasets to show that, 1) the proposed method not only achieves superior performance on compressed videos with high-quality videos pair, 2) it also generalizes well on novel data with only compressed videos available, which implies the promising potential for real-world applications.
Subfields: Biomedical Engineering, Cardiology and Cardiovascular Medicine, Radiology, Nuclear Medicine and Imaging
View Paper: https://www.semanticscholar.org/paper/97c4a50bb344575705703431ee1114279a717a59
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Biomedical Engineering
In Cardiology and Cardiovascular Medicine
In Radiology, Nuclear Medicine and Imaging
Citations in the corpus
- PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer (2021, unknown): “In the rPPG measurement task, the supervision signals from temporal domain (e.g., mean square error loss [10], negative Pearson loss [64, 66]) and frequency domain (e.g., cross-entropy loss [45,62], signal-to-noise ratio loss [52]) provide different extents of constraints for model learning.”
- Face Anti-Spoofing with Human Material Perception (2020, methodology): “Meanwhile, several video-level CNN methods are presented to exploit the dynamic spatio-temporal [9,33,34,35] or rPPG [13,14,12,36,37,38] features for PAD.”