HumanPlus: Humanoid Shadowing and Imitation from Humans
By Zipeng Fu, Qingqing Zhao, Qi Wu, Gordon Wetzstein, Chelsea Finn. Conference on Robot Learning, 2024
331 citations recorded by the source corpus.
One of the key arguments for building robots that have similar form factors to human beings is that we can leverage the massive human data for training. Yet, doing so has remained challenging in practice due to the complexities in humanoid perception and control, lingering physical gaps between humanoids and humans in morphologies and actuation, and lack of a data pipeline for humanoids to learn autonomous skills from egocentric vision. In this paper, we introduce a full-stack system for humanoids to learn motion and autonomous skills from human data. We first train a low-level policy in simulation via reinforcement learning using existing 40-hour human motion datasets. This policy transfers to the real world and allows humanoid robots to follow human body and hand motion in real time using only a RGB camera, i.e. shadowing. Through shadowing, human operators can teleoperate humanoids to collect whole-body data for learning different tasks in the real world. Using the data collected, we then perform supervised behavior cloning to train skill policies using egocentric vision, allowing humanoids to complete different tasks autonomously by imitating human skills. We demonstrate the system on our customized 33-DoF 180cm humanoid, autonomously completing tasks such as wearing a shoe to stand up and walk, unloading objects from warehouse racks, folding a sweatshirt, rearranging objects, typing, and greeting another robot with 60-100% success rates using up to 40 demonstrations. Project website: https://humanoid-ai.github.io/
Subfields: Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/05e61e2a23889323b5397232024467ee5980acde
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
Citations in the corpus
- Open-TeleVision: Teleoperation with Immersive Active Visual Feedback (2024, unknown): “This is also the major innovation that differentiates our work from concurrent works on humanoid teloperation [57, 58].”
- CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models (2025, unknown): “Additionally, inspired by recent advances in robot learning [8, 14, 71], we predict sequences of actions (action chunking) rather than a single action at each timestep.”
- Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes (2024, background): “…compared to other types of robots, including wheeled robots with arms (170, 163, 167, 164, 152, 160, 161, 171, 168), quadrupeds with arms (153, 154, 159, 157), humanoids (155), and even quadrupeds using their legs for both locomotion and manipulation, i.e., loco-manipulation (158, 162, 169, 166).”