DLow: Diversifying Latent Flows for Diverse Human Motion Prediction

By Ye Yuan, Kris M. Kitani. European Conference on Computer Vision, 2020

302 citations recorded by the source corpus.

Deep generative models are often used for human motion prediction as they are able to model multi-modal data distributions and characterize diverse human behavior. While much care has been taken into designing and learning deep generative models, how to efficiently produce diverse samples from a deep generative model after it has been trained is still an under-explored problem. To obtain samples from a pretrained generative model, most existing generative human motion prediction methods draw a set of independent Gaussian latent codes and convert them to motion samples. Clearly, this random sampling strategy is not guaranteed to produce diverse samples for two reasons: (1) The independent sampling cannot force the samples to be diverse; (2) The sampling is based solely on likelihood which may only produce samples that correspond to the major modes of the data distribution. To address these problems, we propose a novel sampling method, Diversifying Latent Flows (DLow), to produce a diverse set of samples from a pretrained deep generative model. Unlike random (independent) sampling, the proposed DLow sampling method samples a single random variable and then maps it with a set of learnable mapping functions to a set of correlated latent codes. The correlated latent codes are then decoded into a set of correlated samples. During training, DLow uses a diversity-promoting prior over samples as an objective to optimize the latent mappings to improve sample diversity. The design of the prior is highly flexible and can be customized to generate diverse motions with common features (e.g., similar leg motion but diverse upper-body motion). Our experiments demonstrate that DLow outperforms state-of-the-art baseline methods in terms of sample diversity and accuracy. Our code is released on the project page: this https URL.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/3dd5fcec294ee8e5693dd100b6c9f4a94c1a0ba3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Behavior-Driven Synthesis of Human Dynamics (2021, methodology): “4 we provide APD comparisons with various motion synthesis approaches.”
GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping (2021, methodology): “However, even recent methods [38, 46, 61, 64], mostly study the body “in isolation”; i.e., with no scene context.”
TEMOS: Generating diverse human motions from textual descriptions (2022, background): “While there is a large body of work focusing on future human motion prediction [3, 6, 16, 37, 54, 57] and completion [10, 17], here, we give an overview of methods that generate motions from scratch (i.”
TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts (2022, background): “This has been traditionally modeled in one-to-one mapping fashion until recent works [3,58,29] that take account the stochastic nature of human motion dynamics.”
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting (2021, methodology): “On the nuScenes dataset, we evaluate our approach against state-of-the-art vehicle prediction meth-ods – Trajectron++ [45], MTP [8], MultiPath [5] Net [39], DSF-AF [33], and DLow-AF [59].”
Action-Conditioned 3D Human Motion Synthesis with Transformer VAE (2021, background): “DLow [67] focuses on diversifying the sampling of future motions from a pretrained model.”
MotionGPT: Human Motion as a Foreign Language (2023, unknown): “Previous research on human motion has explored various tasks, including motion generation [29, 10, 46, 52, 57], motion captioning [9, 11], and motion prediction [56, 61, 24].”