Behavior-Driven Synthesis of Human Dynamics
By Andreas Blattmann, Timo Milbich, Michael Dorkenwald, Björn Ommer. 2021
16 citations recorded by the source corpus.
Generating and representing human behavior are of major importance for various computer vision applications. Commonly, human video synthesis represents behavior as sequences of postures while directly predicting their likely progressions or merely changing the appearance of the depicted persons, thus not being able to exercise control over their actual behavior during the synthesis process. In contrast, controlled behavior synthesis and transfer across individuals requires a deep understanding of body dynamics and calls for a representation of behavior that is independent of appearance and also of specific postures. In this work, we present a model for human behavior synthesis which learns a dedicated representation of human dynamics independent of postures. Using this representation, we are able to change the behavior of a person depicted in an arbitrary posture, or to even directly transfer behavior observed in a given video sequence. To this end, we propose a conditional variational framework which explicitly disentangles posture from behavior. We demonstrate the effectiveness of our approach on this novel task, evaluating capturing, transferring, and sampling fine-grained, diverse behavior, both quantitatively and qualitatively. Project page is available at https://cutt.ly/5l7rXEp
Subfields: Computer Vision and Pattern Recognition, Generative Models
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Björn Ommer
- Andreas Blattmann
- Robin Rombach
- Luc Van Gool
- Ming-Hsuan Yang
- Chen Change Loy
- Dacheng Tao
- Timo Milbich
In Generative Models
- Andreas Blattmann
- Björn Ommer
- Robin Rombach
- Michael Dorkenwald
- Timo Milbich
- Konstantinos G. Derpanis
- Ying Shan
- Chen Change Loy
Citations in the corpus
- iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesis (2021, background): “on a variable number Nc ∈ [1, 5] of local pokes.”
- Stochastic Image-to-Video Synthesis using cINNs (2021, background): “[5] Andreas Blattmann, Timo Milbich, Michael Dorkenwald, and Björn Ommer.”