Is imitation learning the route to humanoid robots?

By Stefan Schaal. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

One-Shot Imitation Learning (2017, methodology): “Survey articles include (Schaal, 1999; Calinon, 2009; Argall et al., 2009).”
Imitating Interactive Intelligence (2020, methodology): “Imitation learning has been successfully deployed to build agents for self-driving cars (Pomerleau, 1989), robotics and biomimetic motor control (Schaal, 1999), game play (Silver et al., 2016; Vinyals et al., 2019), and language modeling (Shannon, 1951).”
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning (2010, methodology): “…are used to learn a controller, have proven very useful in practice and have led to state-of-the art performance in a variety of applications (Schaal, 1999; Abbeel and Ng, 2004; Ratliff et al., 2006; Silver et al., 2008; Argall et al., 2009; Chernova and Veloso, 2009; Ross and Bagnell, 2010).”
RUDDER: Return Decomposition for Delayed Rewards (2018, background): “Reinforcement learning tasks have been changed into supervised tasks [108, 8, 112].”
Off-Policy Deep Reinforcement Learning without Exploration (2018, background): “Imitation learning and its variants are well studied problems (Schaal, 1999; Argall et al., 2009; Hussein et al., 2017).”
Programmatically Interpretable Reinforcement Learning (2018, methodology): “This strategy, inspired by imitation learning (Ross et al., 2011; Schaal, 1999), allows us to perform direct policy search in a highly nonsmooth policy space.”
Reinforcement learning in robotics: A survey (2013, background): “In imitation learning, an agent is provided demonstrations of actions of a good strategy to follow in given situations (Argall et al., 2009; Schaal, 1999).”
Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations (2020, background): “Imitation learning has been applied to multiple domains such as playing table tennis [22], Go [37] and video games [33], and driving autonomously [4, 31] In robotics, the idea of robots learning from human demonstrations have been extensively explored [36, 17, 23, 11, 14, 3, 5].”
Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction (2019, neutral): “This is distinct from previous work (e.g., BCQ [13]) which implicitly constrains the distribution of the learned policy to be close to the behavior policy, similarly to behavioral cloning [32].”
Hyperparameter Selection for Imitation Learning (2021, background): “One of the approaches to overcome this issue is Imitation Learning (IL), which relies on a set of demonstrations presenting the desired behaviour instead of a reward signal (Schaal, 1999; Argall et al., 2009).”
Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control (2021, background): “Imitation learning (IL) has been widely studied in the context of robotic manipulation tasks [2, 4, 5, 11, 31].”
Leveraging Human Guidance for Deep Reinforcement Learning Tasks (2019, background): “Most imitation learning or learning from demonstration algorithms assume this type of guidance [ Schaal, 1999; Argall et al. , 2009; Osa et al. , 2018].”
Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation (2017, methodology): “Imitation learning is a class of methods for acquiring skills by observing demonstrations (see, e.g., [1], [2], [3] for surveys).”
Learning monocular reactive UAV control in cluttered natural environments (2012, methodology): “These data-driven approaches allow us to directly learn a control strategy that mimics an expert pilot’s choice of actions based on demonstrations of the desired behavior, i.e., sample flight trajectories through cluttered environments.”
Third-Person Imitation Learning (2017, background): “Imitation learning has a long history, with several good survey articles, including (Schaal, 1999; Calinon, 2009; Argall et al., 2009).”
An Algorithmic Perspective on Imitation Learning (2018, neutral): “In recent years, imitation learning has been investigated as a way to efficiently and intuitively program autonomous behavior[Schaal, 1999, Argall et al., 2009, Billard et al., 2008, Billard and Grollman, 2013, Bagnell, 2015, Billard et al., 2016].”
Language-Conditioned Imitation Learning for Robot Manipulation Tasks (2020, background): “Learning robot control policies by imitation [31] is an appealing approach to skill acquisition and has been successfully applied to several tasks, including locomotion, grasping, and even table tennis [8, 2, 25].”
Zero-Shot Visual Imitation (2018, background): “The current dominant paradigm in learning from demonstration (LfD) (Argall et al., 2009; Ng & Russell, 2000; Pomerleau, 1989; Schaal, 1999) requires the expert to either manually move the robot joints (i.e., kinesthetic teaching) or teleoperate the robot to execute the desired task.”