Learning from Limited Demonstrations

By Beomjoon Kim, Amir-massoud Farahmand, Joelle Pineau, Doina Precup. 2013

123 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Meta & Continual Learning

Citations in the corpus

Observe and Look Further: Achieving Consistent Performance on Atari (2018, methodology): “More recent approaches have explicitly combined expert demonstrations with RL data during the learning of the policy or action-value function [3, 11, 15].”
Accelerating Online Reinforcement Learning with Offline Datasets (2020, background): “sidered the special case of learning from demonstration data. One class of algorithms initializes the policy via behavioral cloning from demonstrations, and then ﬁne-tunes with reinforcement learning [33, 15, 40, 18, 35, 10, 50]. Most such methods use on-policy ﬁnetuning, which is less sample-efﬁcie”
Off-Policy Deep Reinforcement Learning without Exploration (2018, methodology): “Imitation has been combined with reinforcement learning, via learning from demonstrations methods (Kim et al., 2013; Piot et al., 2014; Chemali & Lazaric, 2015), with deep reinforcement learning extensions (Hester et al.”
Deep Q-learning From Demonstrations (2017, methodology): “This framework was appropriately called Reinforcement Learning with Expert Demonstrations (RLED) in (Piot, Geist, and Pietquin 2014a) and is also evaluated in (Kim et al. 2013; Chemali and Lezaric 2015).”
Learning Self-Correctable Policies and Value Functions from Demonstrations with Negative Sampling (2019, methodology): “Demonstrations have been widely used to improve the efficiency of RL [Kim et al., 2013, Chemali and Lazaric, 2015, Piot et al., 2014], and a common paradigm for continuous state and action space is to initialize with RL algorithms with a good policy or Q function [Rajeswaran et al., 2017, Nair et…”
A Minimalist Approach to Offline Reinforcement Learning (2021, methodology): “…in a variety of manners, such as mixed with adversarial methods [Zhu et al., 2018, Kang et al., 2018], used for pre-training [Pfeiffer et al., 2018], modifying the replay buffer [Večerík et al., 2017, Gulcehre et al., 2020] or adjusting the value function [Kim et al., 2013, Hester et al., 2017].”
Hyperparameter Selection for Imitation Learning (2021, background): “Although expert trajectories can also be useful in this case, this is not the setting of IL, but of RL with demonstrations where both the expert demonstrations and reward signals are used (Kim et al., 2013; Piot et al., 2014; Hester et al., 2018).”
Acme: A Research Framework for Distributed Reinforcement Learning (2020, background): “Promising recent attempts at overcoming exploration in such tasks leverage demonstrations of successful trajectories (Kim et al., 2013; Hester et al., 2017; Vecerik et al., 2017; Nair et al., 2018; Gulcehre et al., 2020), dubbed the RL with Expert Demonstrations (RLED; Piot et al., 2014).”
Reinforcement Learning from Imperfect Demonstrations (2018, methodology): “We follow the Reinforcement Learning with Expert Demonstrations (RLED) framework (Chemali & Lazaric, 2015; Kim et al., 2013; Piot et al., 2014), where both rewards and actions are available in the demonstrations.”
Overcoming Exploration in Reinforcement Learning with Demonstrations (2017, methodology): “Prior work has previously explored improving beyond the demonstrator policy in simple environments by introducing slack variables [35], but our method uses a”
Learning from Demonstrations for Real World Reinforcement Learning (2017, methodology): “, 2015), or to use policy iteration from demonstrations (Kim et al., 2013; Chemali & Lezaric, 2015).”
Reward Augmented Maximum Likelihood for Neural Structured Prediction (2016, background): “An interesting work is [15] which explicitly incorporates supervision in the policy evaluation phase of a policy iteration procedure that otherwise seeks to maximize task reward.”
Making Efficient Use of Demonstrations to Solve Hard Exploration Problems (2019, background): “Many other works have shown that demonstrations can accelerate learning and address hard-exploration tasks (e.g. see Hester et al., 2018; Kim et al., 2013; Nair et al., 2018; Kang et al., 2018).”
Imitation Learning as f-Divergence Minimization (2019, background): “[18] Beomjoon Kim, Amir-massoud Farahmand, Joelle Pineau, and Doina Precup.”
Model-Free Imitation Learning with Policy Optimization (2016, background): “When either of these additional resources is provided, alternative approaches (Kim et al., 2013; Daum ´ e III et al., 2009; Ross & Bagnell, 2010; Ross et al., 2011) may be more sample efﬁcient, and investigating ways to combine these resources with our framework is an interesting research direction.”
Transfer Learning in Deep Reinforcement Learning: A Survey (2020, methodology): “Another example is the Approximate Policy Iteration with Demonstration (APID) algorithm, which was proposed by [59] and extended by [95].”
Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration (2018, unknown): “A common countermeasure in RL is to pre-train the agent to mimic expert demonstrations via behavioral cloning (Pomerleau, 1991; Kim et al., 2013), encouraging it to take similar actions in similar states.”
Recent Advances in Robot Learning from Demonstration (2020, background): “In an effort to leverage the benefits of LfD’s data efficiency, researchers have demonstrated that LfD can be combined with reinforcement learning in order to improve sample efficiency (189, 190, 191, 192, 193, 194, 195, 196).”