Hierarchical Relative Entropy Policy Search

By Christian Daniel, Gerhard Neumann, Jan Peters. 2014

252 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Learning Dynamic Robot-to-Human Object Handover from Human Feedback (2016, background): “To allow robot skills to adapt to different situations, contextual policy search learns a contextual policy that conditions a skill on context variables [8, 9, 18].”
Reinforcement Learning with Deep Energy-Based Policies (2017, neutral): “Multi-modality also has application in real robot tasks, as demonstrated in (Daniel et al., 2012).”
Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations (2020, background): “Previous approaches have looked at how to enforce diversity in an agent’s behavior [12, 15, 9].”
Maximum a Posteriori Policy Optimisation (2018, methodology): “A number of other authors have considered the same perspective but in a model-free RL setting (Neumann, 2011; Peters et al., 2010a; Florensa et al., 2017; Daniel et al., 2016) or inverse RL problems (Ziebart et al., 2008).”
RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, background): “…et al., 2018; Sukhbaatar et al., 2018; Dwiel et al., 2019; Nair & Finn, 2019; Nasiriany et al., 2019; Dilokthanakul et al., 2019) or by discovering reusable skills (Daniel et al., 2012; Gregor et al., 2016; Warde-Farley et al., 2018; Shankar & Gupta, 2020; Thomas et al., 2018; Sharma et al., 2020).”
Modular Multitask Reinforcement Learning with Policy Sketches (2016, background): “Techniques for learning options with less guidance than the present work include Bacon & Precup (2015) and Vezhn-evets et al. (2016), and other general hierarchical policy learners include Daniel et al. (2012), Bakker & Schmidhu-ber (2004) and Menache et al. (2002).”
Collective robot reinforcement learning with distributed asynchronous guided policy search (2016, methodology): “The constraint ensures that the local policies only change by a small amount at each iteration, to prevent divergence of the trajectory-centric RL algorithm, analogously to other recent RL methods [2, 23].”
Dynamics-Aware Unsupervised Discovery of Skills (2019, background): “This principle, proposed in prior work that utilized purely model-free unsupervised RL methods [14, 18, 17, 24, 71], aims to learn diverse skills via a discriminability objective: a good set of skills is one where it is easy to distinguish the skills from each other, which means they perform distinc”
Data-Efficient Hierarchical Reinforcement Learning (2018, background): “These approaches rely on hand-crafted rewards based on prior domain knowledge [21, 18, 22, 40] or diversity-encouraging rewards like mutual information [6, 10].”
Stochastic Neural Networks for Hierarchical Reinforcement Learning (2016, background): “Another line of work on skill discovery particularly relevant to our approach is the HiREPS al-gorithm by Daniel et al. (2012) where, instead of having a mutual information bonus like in our proposed approach, they introduce a constraint on an equivalent metric.”
Model-Free Preference-Based Reinforcement Learning (2016, methodology): “The policy update of actor critic REPS is inspired by the episodic REPS algorithm (Daniel et al., 2012).”
Learning by Playing - Solving Sparse Reward Tasks from Scratch (2018, background): “…connected to the broader literature on multi-task (reinforcement) learning (see e.g. Caruana (1997) for a general overview and Lazaric et al. (2008); Mehta et al. (2008) for RL applications) and work on reinforcement learning via options (Dietterich, 1998; Bacon et al., 2017; Daniel et al., 2012).”
Language as an Abstraction for Hierarchical Deep Reinforcement Learning (2019, background): “These rewards include mutual information-based diversity rewards [12, 17], hand-crafted rewards based on domain knowledge [29, 24, 30, 59], and goal-oriented rewards [15, 49, 3, 62, 36, 37].”
Latent Space Policies for Hierarchical Reinforcement Learning (2018, background): “Another option is to train the entire hierarchy end-to-end (Bacon et al., 2017; Vezhnevets et al., 2017; Daniel et al., 2012).”
Data-Efficient Generalization of Robot Skills with Contextual Policy Search (2013, methodology): “In the episodic contextual setting, the context s and the parameter ω uniquely determine the trajectory distribution (Daniel et al. 2012).”
Regularized Hierarchical Policies for Compositional Transfer in Robotics (2019, unknown): “Some aspects of it are similar to Daniel et al. [2016] but we phrase the learning problem in a fashion suitable for large-scale off-policy learning with deep neural networks.”