Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning

By Jun Morimoto, Kenji Doya. 2000

269 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Reinforcement learning in robotics: A survey (2013, methodology): “continuous actor-critic method in the lower level (Morimoto and Doya, 2001).”
Full Paper Reinforcement Learning for Imitating Constrained Reaching Movements (unknown): “RL algorithms have already been successfully implemented in robotic applications such as swinging up and controlling an inverse pendulum [7], refining forehand and backhand tennis swing [8] or acquiring a dynamic stand-up behavior [9, 10].”
Hierarchical Relative Entropy Policy Search (2014, methodology): “We treat the hierarchical structure of the policy as a latent variable, which results in the Hierarchical Relative Entropy Policy Search (HiREPS) algorithm.”
A Hierarchy of Time-Scales and the Brain (2008, unknown): “In robotics and motor control theory, conflict resolution among different time-scales has been addressed using hierarchical control structures [17–22].”
MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale (2021, background): “…and composite skills has been addressed through hierarchical reinforcement learning with options [75, 7, 14], combining multiple sub-tasks [6, 16, 49, 26, 69, 89], reusing samples between tasks [39], relabeling experience in hindsight [3], introducing demonstrations [58, 81, 29, 41, 70, 66, 67].”
Generalized Hindsight for Reinforcement Learning (2020, background): “Hierarchical reinforcement learning (Morimoto & Doya, 2001; Barto et al., 2004) is another framework amenable for multitask learning.”
SkyAI: Highly modularized reinforcement learning library (2010, methodology): “config ={ Factor ={ resize(17, 5) [0]= ( 0, 0, 0, 0, 0) [1]= ( 1, 0, 0, 0, 0) [2]= ( 0, 0, 0, 0, 0) [3]= ( 0, 1, 0, 0, 0) [4]= ( 1, 0, 0, 0, 0) [5]= ( 0, 0, 0, 0, 0) [6]= ( 0, 1, 0, 0, 0) [7]= ( 0, 0, 0, 0, 0) [8]= ( 0, 0, 1, 0, 0) [9]= ( 0, 0, 0, 1, 0) [10]= ( 0, 0, 0, 0, 1) [11]= ( 0, 0, 0, 0, 0) ”
Robust Reinforcement Learning (2005, unknown): “Thus the use of environmental models have been quite common both for on-line action planning [3] and for off-line learning by simulation [4].”
Reducing hardware experiments for model learning and policy optimization (2015, background): “Another approach is model-free policy optimization, where the policy is improved through a number of hardware trials [4], [5].”
Hierarchical Policy Gradient Algorithms (2003, methodology): “We accelerate the learning of HPG algorithms by formulating higher-level subtasks, which usually involve smaller and more manageable state and finite action spaces, as VFRL problems, and lower-level subtasks with infinite state and/or action spaces as PGRL problems (Morimoto & Doya, 2001).”
Q-Cut - Dynamic Discovery of Sub-goals in Reinforcement Learning (2002, methodology): “The distinctive empirical results motivate the application of the Q-Cut algorithm to a variety of problems where bottlenecks may arise.A car parking problem, a robot learning to stand up (see [ 13 ]), and some scheduling problems, are characterized by the existence of bottlenecks that must be reache”
Composing Meta-Policies for Autonomous Driving Using Hierarchical Deep Reinforcement Learning (2017, methodology): “The HRL approach for policy composition has been applied since the 1990s [12, 13, 3, 14, 17, 20].”
Learning CPG-based biped locomotion with a policy gradient method (2005, methodology): “In addition, practical feasibility of this function approximator was demonstrated for real-time implementation of the control policy on a hardware robot to achieve the desired behavior [18].”
Metalearning and neuromodulation (2002, background): “The theory has been successfully applied to a variety of dynamic optimization problems, such as game programs (Tesauro, 1994), robotic control (Morimoto & Doya, 2001), and resource allocation (Singh & Bertsekas, 1997).”