Perpetual Humanoid Control for Real-time Simulated Avatars

By Zhengyi Luo, Jinkun Cao, Alexander W. Winkler, Kris M. Kitani, Weipeng Xu. IEEE International Conference on Computer Vision, 2023

307 citations recorded by the source corpus.

We present a physics-based humanoid controller that achieves high-fidelity motion imitation and fault-tolerant behavior in the presence of noisy input (e.g. pose estimates from video or generated from language) and unexpected falls. Our controller scales up to learning ten thousand motion clips without using any external stabilizing forces and learns to naturally recover from fail-state. Given reference motion, our controller can perpetually control simulated avatars without requiring resets. At its core, we propose the progressive multiplicative control policy (PMCP), which dynamically allocates new network capacity to learn harder and harder motion sequences. PMCP allows efficient scaling for learning from large-scale motion databases and adding new tasks, such as fail-state recovery, without catastrophic forgetting. We demonstrate the effectiveness of our controller by using it to imitate noisy poses from video-based pose estimators and language-based motion generators in a live and real-time multi-person avatar use case.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/1fbdd5a3ba7d17ab9c24c8c0e6ea93d7ccb90c16

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

  1. Tairan He
  2. Zhengyi Luo

In Control and Systems Engineering

  1. Tairan He
  2. Zhengyi Luo

Citations in the corpus