Mode-adaptive neural networks for quadruped motion control

By He Zhang, Sebastian Starke, Taku Komura, Jun Saito. 2018

370 citations recorded by the source corpus.

Quadruped motion includes a wide variation of gaits such as walk, pace, trot and canter, and actions such as jumping, sitting, turning and idling. Applying existing data-driven character control frameworks to such data requires a significant amount of data preprocessing such as motion labeling and alignment. In this paper, we propose a novel neural network architecture called Mode-Adaptive Neural Networks for controlling quadruped characters. The system is composed of the motion prediction network and the gating network. At each frame, the motion prediction network computes the character state in the current frame given the state in the previous frame and the user-provided control signals. The gating network dynamically updates the weights of the motion prediction network by selecting and blending what we call the expert weights, each of which specializes in a particular movement. Due to the increased flexibility, the system can learn consistent expert weights across a wide range of non-periodic/periodic actions, from unstructured motion capture data, in an end-to-end fashion. In addition, the users are released from performing complex labeling of phases in different gaits. We show that this architecture is suitable for encoding the multi-modality of quadruped locomotion and synthesizing responsive motion in real-time.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning (2019, methodology): “Motion clips are collected from publicly available datasets (CMU; SFU; Zhang et al., 2018).”
Policies Modulating Trajectory Generators (2018, methodology): “In computer graphics, special architectures, such as Phase-Functioned Neural Networks [11] and Mode-Adaptive Neural Networks [12], have been proposed to synthesize locomotion controllers from motion capture data.”
AMP (2021, methodology): “Detailed hyperparameter settings are available in Appendix B. Reference motion clips are collected from a combination of public mocap libraries, custom recorded mocap clips, and artist-authored keyframe animations [CMU [n.d.]; SFU [n.d.]; Zhang et al. 2018].”
Hierarchical Style-based Networks for Motion Synthesis (2020, methodology): “ataset. Meanwhile, it is hard for these approaches to generalize to unseen distribution of data. To adapt and generalize to large-scale data, deep neural networks are utilized in several recent works [11,10,41]. For example, Holden et al. [11] introduced to synthesize the motion sequence using deep ”
Learning Agile Robotic Locomotion Skills by Imitating Animals (2020, methodology): “The mocap clips are collected from a public dataset [68] and retargeted to the Laikago following the procedure in Section IV.”
AI Choreographer: Music Conditioned 3D Dance Generation with AIST++ (2021, background): “Phase-functioned neural networks and it’s variations [94, 33, 73, 74] address this issue via conditioning the network weights on phase, however, they do not scale well to represent a wide variety of motion.”
Character controllers using motion VAEs (2020, methodology): “We use a style of MoE proposed in [Zhang et al. 2018], which we empirically observe to help achieve slightly better pose construction and reduced visual artifacts.”