Exploration and apprenticeship learning in reinforcement learning

By Pieter Abbeel, Andrew Y. Ng. 2005

283 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Networks and Communications, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Reinforcement Learning

Citations in the corpus

Concrete Problems in AI Safety (2016, background): “…in that the former involves attacks against a legitimate ML system by an adversary (e.g. a criminal tries to fool a face recognition system), while the latter involves attacks by an ML system controlled by an adversary (e.g. a criminal trains a “smart hacker” system to break into a website).”
Learning Parameterized Skills (2012, methodology): “Duff [21] proposes a Bayesian approach for the case in which prior uncertainties about the transition probabilities are available; Abbeel and Ng [1] present a method for computing a near-optimal policy assuming that demonstrations by a teacher are available.”
Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, background): “In [67] it is shown that given the initial demonstration, no explicit exploration is necessary, and we can attain near-optimal performance.”
Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning (2011, background): “, by imitation [3]), or a deep understanding of the system.”
Improving Multi-Step Prediction of Learned Time Series Models (2015, methodology): “The final dynamical system we test on is the simulated helicopter from (Abbeel and Ng 2005a).”
Agnostic System Identification for Model-Based Reinforcement Learning (2012, positive): “We demonstrate the efﬁcacy of DAgger on a challenging problem: learning to perform aerobatic maneuvers with a simulated helicopter, using the simulator of Abbeel & Ng (2005), which has a continuous 21-dimensional state and 4-dimensional control space.”
Preparing for the Unknown: Learning a Universal Policy with Online System Identification (2017, methodology): “For example, Abbeel and Ng [3] used a time-variant linear function to model the dynamics of a helicopter from the realworld data while learning a control policy to perform inverted autonomous helicopter flight.”
Autonomic multi-agent management of power and performance in data centers (2008, unknown): “Other related approaches which we expect to be of benefit include model-based reinforcement learning approaches, such as Apprenticeship Learning [2] (Abbeel and Ng, 2005) and Differential Dynamic Programming [1], which uses simultaneous learning state-transition models along with expected value func”
A Practical and Conceptual Framework for Learning in Control (2010, methodology): “To overcome the problem of policies for inaccurate models, Abbeel et al. (2006) added a bias term to the model when updating the model after gathering real experience. Poupart and Vlassis (2008) learned a probabilistic model for a finite-state POMDP to incorporate observations into prior knowledge. ”
CORL: A Continuous-state Offset-dynamics Reinforcement Learner (2008, methodology): “From Abbeel and Ng (2005) (Proposition 7), this expression is upperbounded by ||β2 + s − (β1 + s)||2√ 2πσmin = ||β2 + β1||2√ 2πσmin .”
Learning and exploration in action-perception loops (2013, background): “Recent research has attempted to decouple the exploration and exploitation components of optimal control in multi-armed bandits (Abbeel and Ng, 2005; Bubeck et al., 2009).”
Knows what it knows: a framework for self-aware learning (2008, methodology): “Our model formulation is based on Strehl and Littman (2008), and is slightly different from Abbeel and Ng (2005). Here, S ⊆ nS and A ⊆ nA are the state and action spaces, respectively.”
Transfer learning for reinforcement learning on a physical robot (2010, background): “sufficient to teach an agent to control an autonomous helicopter [1, 4].”
Bounded Optimal Exploration in MDP (2016, background): “This problem is a slightly generalized version of those considered by Abbeel and Ng (2005), Strehl and Littman (2008b), and Li et al. (2011).”
A survey of robot learning from demonstration (2009, unknown): “In addition to updating state values, student execution informationmay be used to update the actual learned dynamics model T (s|s, a), as in work that seeds the world model with teacher demonstrations [97].”
Active reinforcement learning (2008, background): “Inthis paper, we propose another alternative: given initial (possibly inaccurate) speci.cation of the MDP, the agent determines the sensitivity of the optimalpolicy to changesin transitions and rewards.”