DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks
By Weixuan 'Vincent' Chen, Daniel J. McDuff. European Conference on Computer Vision, 2018
647 citations recorded by the source corpus.
Non-contact video-based physiological measurement has many applications in health care and human-computer interaction. Practical applications require measurements to be accurate even in the presence of large head rotations. We propose the first end-to-end system for video-based measurement of heart and breathing rate using a deep convolutional network. The system features a new motion representation based on a skin reflection model and a new attention mechanism using appearance information to guide motion estimation, both of which enable robust measurement under heterogeneous lighting and major motions. Our approach significantly outperforms all current state-of-the-art methods on both RGB and infrared video datasets. Furthermore, it allows spatial-temporal distributions of physiological signals to be visualized via the attention mechanism.
Subfields: Biomedical Engineering, Cardiology and Cardiovascular Medicine, Radiology, Nuclear Medicine and Imaging
View Paper: https://www.semanticscholar.org/paper/1f220beedd8d00627aedb7e4f0bb4ce45c75c65c
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Biomedical Engineering
In Cardiology and Cardiovascular Medicine
In Radiology, Nuclear Medicine and Imaging
Citations in the corpus
- Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks (2019, methodology): “In order to make fair comparison with previous works [3, 12, 18], we follow the same routine as their works and use 30 seconds clip (frames 306 to 2135) of each video.”
- RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation (2019, methodology): “, I3D [38] and DeepPhy [26], are also used for”
- PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer (2021, neutral): “In the rPPG measurement task, the supervision signals from temporal domain (e.g., mean square error loss [10], negative Pearson loss [64, 66]) and frequency domain (e.g., cross-entropy loss [45,62], signal-to-noise ratio loss [52]) provide different extents of constraints for model learning.”
- Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement (2020, methodology): “For our optical basis we start with Shafer’s Dichromatic Reflection Model (DRM), as in prior work [17, 9].”
- FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals (2019, methodology): “Ob-serving subtle color and motion based signals from videos [58, 15] enabled methodologies like remote photoplethys-mography (rPPG or iPPG) [47, 43] and head motion based ballistocardiogram (BCD) [7] for non-intrusive heart rate detection from facial RGB videos.”
- Deep learning-enabled medical computer vision (2021, methodology): “For instance, systems could use video 141 to analyze heart and breathing rates 141 .”
- Remote Heart Rate Measurement From Highly Compressed Facial Videos: An End-to-End Deep Learning Solution With Video Enhancement (2019, methodology): “We follow the same routine as in previous works [17, 25, 3] and use 30 seconds (frames 306 to 2135) of each video.”