Catch & Carry

By Josh Merel, Saran Tunyasuvunakool, Arun Ahuja, Yuval Tassa, Leonard Hasenclever, Vu Pham, Tom Erez, Greg Wayne, Nicolas Manfred Otto Heess. 2019

103 citations recorded by the source corpus.

We address the longstanding challenge of producing flexible, realistic humanoid character controllers that can perform diverse whole-body tasks involving object interactions. This challenge is central to a variety of fields, from graphics and animation to robotics and motor neuroscience. Our physics-based environment uses realistic actuation and first-person perception - including touch sensors and egocentric vision - with a view to producing active-sensing behaviors (e.g. gaze direction), transferability to real robots, and comparisons to the biology. We develop an integrated neural-network based approach consisting of a motor primitive module, human demonstrations, and an instructed reinforcement learning regime with curricula and task variations. We demonstrate the utility of our approach for several tasks, including goal-conditioned box carrying and ball catching, and we characterize its behavioral robustness. The resulting controllers can be deployed in real-time on a standard PC.1

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Accelerating Reinforcement Learning with Learned Skill Priors (2020, background): “Prior work has learned skill libraries from data collected by humans [27, 22, 23, 29, 19] or by agents autonomously exploring the world [12, 30].”
Perpetual Humanoid Control for Real-time Simulated Avatars (2023, background): “Governed by the laws of physics, simulated characters [30, 29, 31, 33, 32, 6, 43, 50, 26, 12, 2, 10, 44, 11] have the distinct advantage of creating natural human motion, human-to-human interaction [18, 46], and human-object interactions [26, 32].”
GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping (2021, background): “Merel et al. [42] use deep RL and human MoCap demonstrations to learn a vision-guided neural controller for picking up and carrying boxes, or catching/throwing a ball.”
From Motor Control to Team Play in Simulated Humanoid Football (2021, methodology): “It extends the environment suite of (23, 31, 33, 42) and comprises teams of fully articulated humanoid”
Skill-based Meta-Reinforcement Learning (2022, background): “When trained from diverse datasets these approaches can extract a wide repertoire of skills and learn complex, long-horizon tasks (Merel et al., 2020; Lynch et al., 2020; Pertsch et al., 2020; Ajay et al., 2021; Chebotar et al., 2021; Pertsch et al., 2021).”