EDGE: Editable Dance Generation From Music
By Jo-Han Tseng, Rodrigo Castellon, C. Karen Liu. Computer Vision and Pattern Recognition, 2022
404 citations recorded by the source corpus.
Dance is an important human art form, but creating new dances can be difficult and time-consuming. In this work, we introduce Editable Dance GEneration (EDGE), a state-of-the-art method for editable dance generation that is capable of creating realistic, physically-plausible dances while remaining faithful to the input music. EDGE uses a transformer-based diffusion model paired with Jukebox, a strong music feature extractor, and confers powerful editing capabilities well-suited to dance, including joint-wise conditioning, and in-betweening. We introduce a new metric for physical plausibility, and evaluate dance quality generated by our method extensively through (1) multiple quantitative metrics on physical plausibility, beat alignment, and diversity benchmarks, and more importantly, (2) a large-scale user study, demonstrating a significant improvement over previous state-of-the-art methods. Qualitative samples from our model can be found at our website.
Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Signal Processing
View Paper: https://www.semanticscholar.org/paper/465b80aea2b99ffe0710325f5c256e8d2ac642a5
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
In Control and Systems Engineering
In Signal Processing
Citations in the corpus
- MoMask: Generative Masked Modeling of 3D Human Motions (2023, background): “Recently, we have witnessed the surge of works for neural motion generation, with conditioning on various domains such as motion prefix [29, 33], action class [6, 14, 31, 35], audio [13, 40, 43, 53], texts [9, 15, 16, 36, 42].”
- Human Motion Diffusion as a Generative Prior (2023, methodology): “EDGE [Tseng et al. 2022] suggested the unfolding method”
- Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation (2023, methodology): “We adopt the temporal aggregation method in [43], connecting re-sults from different batches to generate long videos.”
- Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance (2024, methodology): “To accommodate the input of a video clip comprising 24 frames, a temporal aggregation technique [41] is employed to concatenate multiple clips.”