Eulerian video magnification for revealing subtle changes in the world

By Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John V. Guttag, Frédo Durand, William T. Freeman. 2012

1,524 citations recorded by the source corpus.

Our goal is to reveal temporal variations in videos that are difficult or impossible to see with the naked eye and display them in an indicative manner. Our method, which we call Eulerian Video Magnification, takes a standard video sequence as input, and applies spatial decomposition, followed by temporal filtering to the frames. The resulting signal is then amplified to reveal hidden information. Using our method, we are able to visualize the flow of blood as it fills the face and also to amplify and reveal small motions. Our technique can run in real time to show phenomena occurring at the temporal frequencies selected by the user.

Subfields: Artificial Intelligence, Biomedical Engineering, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Signal Processing

Citations in the corpus

Learning-based Video Motion Magnification (2018, methodology): “While previous techniques rely on hand-crafted ﬁlters [26, 30], our goal is to learn a set of ﬁlters that extracts and manipulates representations of the motion signal δ ( x , t ) to generate output magniﬁed frames.”
Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks (2016, background): “…Instead of ﬁnding an intrinsic representation of the image itself, as most previous work has done [Radford et al., 2016, Reed et al., 2015], our network ﬁnds an intrinsic representation of intensity changes between two images, also known as the difference image or Eulerian motion [Wu et al., 2012].”
Detecting Pulse from Head Motions in Video (2013, background): “Our work is also inspired by the amplification of imperceptible motions in video [22, 10].”
Towards Reading Hidden Emotions: A Comparative Study of Spontaneous Micro-Expression Spotting and Recognition Methods (2015, methodology): “The magnification process is implemented using codes shared by authors of [46].”
FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals (2019, methodology): “Ob-serving subtle color and motion based signals from videos [58, 15] enabled methodologies like remote photoplethys-mography (rPPG or iPPG) [47, 43] and head motion based ballistocardiogram (BCD) [7] for non-intrusive heart rate detection from facial RGB videos.”
Face Spoof Detection With Image Distortion Analysis (2015, background): “For example , in [12], the authors showed that appropriately magnified motion cue [23] improves the performance of texture based approaches (HTER = 6.62% on the Idiap database with motion magnification compared to HTER = 11.75% without motion magnification, both using LBP features).”
Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward (2021, methodology): “Initially, three techniques: skin color variation [99], average optical intensity [100], and Eulerian video magnification [101], are used to measure heart rate.”
Ten simple rules for responsible big data research (2017, background): “, it is possible to extract data on the exact heart rates of people from YouTube videos [2].”
MoSh (2014, background): “There has been recent work on magnifying small motions in video sequences [Wang et al. 2007; Wu et al. 2012; Wadhwa et al. 2013] but less work on magnifying 3D motions.”
RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation (2019, methodology): “Magnified skin color changes due to heartbeat using Eulerian Video Magnification [29].”
DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks (2018, background): “Computer analysis of videos and images can be used to recover these signals [30, 24, 3] and magnify them [40, 14].”
Smart Homes that Monitor Breathing and Heart Rate (2015, methodology): “The figure shows the output of the FFT after applying a hanning window and filtering between [40-200] beats/minute.”
Video magnification in presence of large motions (2015, methodology): “Lagrangian approaches can only magnify motion changes [7, 2], while Eulerian can magnify motion [14, 11] as well as color changes [17].”
Riesz pyramids for fast phase-based video magnification (2014, methodology): “In practice, we found that using two three-tap filters to approximate the Riesz transform gave motion magnification results that were comparable to using higher-tap approximations or the frequency domain implementation of the Riesz transform.”
A Boost in Revealing Subtle Facial Expressions: A Consolidated Eulerian Framework (2019, methodology): “such as the Eulerian based MAG method [30] and the graph spectral TIM method [2], existing MER approaches [34], [11], [14] usually treat MAG and TIM as two separate modules and adopt them in a successive manner.”
Non-contact video-based vital sign monitoring using ambient light and auto-regressive models (2014, background): “A second MIT group has been developing Eulerian video magniﬁcation techniques to track and amplify the motion-related changes caused by breathing in the videos of human subjects, including neonates, recorded under normal lighting conditions (Wu et al 2012).”