Phase-based video motion processing

By Neal Wadhwa, Michael Rubinstein, Frédo Durand, William T. Freeman. 2013

867 citations recorded by the source corpus.

We introduce a technique to manipulate small movements in videos based on an analysis of motion in complex-valued image pyramids. Phase variations of the coefficients of a complex-valued steerable pyramid over time correspond to motion, and can be temporally processed and amplified to reveal imperceptible motions, or attenuated to remove distracting changes. This processing does not involve the computation of optical flow, and in comparison to the previous Eulerian Video Magnification method it supports larger amplification factors and is significantly less sensitive to noise. These improved capabilities broaden the set of applications for motion processing in videos. We demonstrate the advantages of this approach on synthetic and natural video sequences, and explore applications in scientific analysis, visualization and video enhancement.

Subfields: Atomic and Molecular Physics, and Optics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atomic and Molecular Physics, and Optics

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning-based Video Motion Magnification (2018, methodology): “The Eulerian approaches [5,1,6], on the other hand, decompose video frames into representations that facilitate manipulation of motions, without requiring explicit tracking.”
Video Frame Interpolation via Adaptive Convolution (2017, methodology): “These methods encode motion in the phase difference between input frames and manipulate the phase information for applications like motion magnification [51] and view expansion [6].”
Advances in Computer Vision-Based Civil Infrastructure Inspection and Monitoring (2019, methodology): “Several algorithms are available to accomplish displacement extraction; these work in principle by template matching or by tracking either the contours of the constant phase or intensity through time [55,197,198].”
Riesz pyramids for fast phase-based video magnification (2014, methodology): “These motions require computational amplification to be revealed [10, 12, 19, 21].”
Phase-based frame interpolation for video (2015, methodology): “Recently, phase-based methods have shown promising results in applications such as motion and view extrapolation [10, 27].”
MoSh (2014, background): “There has been recent work on magnifying small motions in video sequences [Wang et al. 2007; Wu et al. 2012; Wadhwa et al. 2013] but less work on magnifying 3D motions.”
Output-only computer vision based damage detection using phase-based optical flow and unscented Kalman filters (2017, methodology): “[6] proposed a displacement measurement methodology from video, inspired by phase-based motion magnification [19].”
Video Acceleration Magnification (2017, methodology): “Tiny temporal variations that are hard or impossible to see with the naked eye can be enhanced by impressive video magnification algorithms [26, 27].”