Reinforcement Learning from Demonstration through Shaping

By Tim Brys, Ani Harutyunyan, Halit Bener Suay, Sonia Chernova, Matthew E. Taylor, Ann Nowé. 2015

242 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Teaching Autoregressive Language Models Complex Tasks By Demonstration (2021, background): “In other words, the look actions help to make the relevant steps of the problem explicit, likely facilitating shaping (cf. [38]).”
Deep Q-learning From Demonstrations (2017, methodology): “Follow-ups to this work showed how expert advice or demonstrations can be used to shape rewards in the RL problem (Brys et al. 2015; Suay et al. 2016).”
Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations (2017, neutral): “Additionally demonstrations have been used to guide exploration through reward/policy shaping but these are often rule-based or work on discrete spaces making them difficult to apply to high dimensional dexterous manipulation [48], [7], [45].”
Watch, Try, Learn: Meta-Learning from Demonstrations and Reward (2019, methodology): “This ranges from simple techniques such as demonstration-based pre-training and initialization [28, 19, 21, 20, 35] to more complex methods that incorporate both demonstration data and reward information in the loop of training [40, 3, 37, 14, 38, 29, 26, 22].”
Learning from Demonstrations for Real World Reinforcement Learning (2017, background): “Follow-ups to this work showed how expert advice or demonstrations can be used to shape rewards in the RL problem (Brys et al., 2015; Suay et al., 2016).”
Mapping Instructions and Visual Observations to Actions with Reinforcement Learning (2017, methodology): “We incorporate complete trajectories by using a simplification of the shaping term introduced by Brys et al. (2015). Unlike F1, it requires access to the previous state and action.”
Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration (2018, background): “Several previous works also explore neighborhoods of demonstrations via reward shaping (Brys et al., 2015; Hussein et al., 2017) or off-policy sampling (Levine & Koltun, 2013).”
Reinforcement Learning with Multiple Experts: A Bayesian Model Combination Approach (2018, methodology): “[2015]), from similar solved tasks (Taylor and Stone [2009]), from demonstrations (Brys et al. [2015]), and in general can be analytic or computational.”
Learning from Demonstration for Shaping through Inverse Reinforcement Learning (2016, methodology): “introduced a new algorithm we refer to as Similarity Based Shaping (SBS) [4].”
Imitation Learning (2017, methodology): “A different approach to combine learning from demonstrations with reinforcement learning is employed in Brys et al. [2015a].”
Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning (2020, unknown): “Demonstration data can be used to derive a policy [15] which might be further reﬁned using typical RL training [13] or to shape the rewards biasing towards previously seen expert demonstrations [4].”
Guided Meta-Policy Search (2019, unknown): “…learning more efﬁcient in the single-task setting (Peters & Schaal, 2006; Kober & Peters, 2009; Kormushev et al., 2010; Taylor et al., 2011; Brys et al., 2015; Subramanian et al., 2016; Hester et al., 2018; Sun et al., 2018; Rajeswaran et al., 2018; Nair et al., 2018; Kober et al., 2013;…”
Transfer Learning in Deep Reinforcement Learning: A Survey (2020, background): “Restrictions apply. i is based on its priority p i with a temperature parameter α : Another algorithm named LfDS was proposed by [96], which draws a close connection to reward shaping (Section V-A).”
Policy Optimization with Demonstrations (2018, background): “Recently, Brys et al. (2015) introduced a reward reshaping mechanism to encourage taking actions close to the demonstrated ones.”