PhysDiff: Physics-Guided Human Motion Diffusion Model

By Ye Yuan, Jiaming Song, Arash Vahdat, Jan Kautz. IEEE International Conference on Computer Vision, 2022

454 citations recorded by the source corpus.

Denoising diffusion models hold great promise for generating diverse and realistic human motions. However, existing motion diffusion models largely disregard the laws of physics in the diffusion process and often generate physically-implausible motions with pronounced artifacts such as floating, foot sliding, and ground penetration. This seriously impacts the quality of generated motions and limits their real-world application. To address this issue, we present a novel physics-guided motion diffusion model (PhysDiff), which incorporates physical constraints into the diffusion process. Specifically, we propose a physics-based motion projection module that uses motion imitation in a physics simulator to project the denoised motion of a diffusion step to a physically-plausible motion. The projected motion is further used in the next diffusion step to guide the denoising diffusion process. Intuitively, the use of physics in our model iteratively pulls the motion toward a physically-plausible space, which cannot be achieved by simple post-processing. Experiments on large-scale human motion datasets show that our approach achieves state-of-the-art motion quality and improves physical plausibility drastically (>78% for all datasets).

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/9d5e20f90fa3e7b126641ac6df5f42478ff7fb24

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation (2023, background): “…in various domains, including images (Dhariwal & Nichol, 2021; Nichol & Dhariwal, 2021; Saharia et al., 2022b; Ramesh et al., 2022; Rombach et al., 2022), video (Ho et al., 2022; Singer et al., 2022), 3D scenes (M ´ uller et al., 2022), and motion sequences (Yuan et al., 2022; Tevet et al., 2022).”
Perpetual Humanoid Control for Real-time Simulated Avatars (2023, background): “Furthermore, to achieve better downstream tasks, the current disjoint process (where the video pose estimator is unaware of the physics simulation) may be insufficient; tighter integration with pose estimation [52, 21] and language-based motion generation [51] is needed.”
Human Motion Diffusion as a Generative Prior (2023, background): “PhysDiff [Yuan et al. 2022] added to MDM a pre-trained physical model based on reinforcement learning which enforces physical constraints during the sampling process.”