Learning to Drive a Bicycle Using Reinforcement Learning and Shaping

By Jette Randløv, Preben Alstrøm. 1998

Subfields: Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

Meta-Gradient Reinforcement Learning (2018, background): “…corrections [Espeholt et al., 2018, Munos et al., 2016], target networks [Mnih et al., 2015], emphasis on certain states [Sutton et al., 2016], reward clipping [Mnih et al., 2013], or even the nature of the rewards themselves [Randløv and Alstrøm, 1998, Singh et al., 2005, Zheng et al., 2018].”
Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, background): “tion F, and r0 is the signal given to the agent by the augmented reward function R0. Empirical evidence has shown that reward shaping can be a powerful tool to improve the learning speed of RL agents [47]. However, it can have unintended consequences. The implication of adding a shaping reward is th”
Reinforcement learning in robotics: A survey (2013, background): “…(1997), Kirchner (1997), Morimoto and Doya (2001), Muelling et al. (2012), and Whitman and Atkeson (2010) Progressive tasks Asada et al. (1996) and Randløv and Alstrøm (1998) Directed exploration with prior knowledge Approach Employed by… Directed exploration Huang and Weng (2002), Kroemer et…”
Increasing the Action Gap: New Operators for Reinforcement Learning (2015, background): “We now study the behaviour of our new operators on the bi-cycle domain (Randlov and Alstrom 1998).”
Q(λ) with Off-Policy Corrections (2016, methodology): “To this end, we applied Q∗(λ) to the Bicycle domain [11].”
Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces (2017, background): “Another area of research that aims to use explicit human interaction to increase the speed at which autonomous agents can learn to perform tasks is that of reward shaping (Skinner 1938; Randløv and Alstrøm 1998; Ng, Harada, and Russell 1999; Devlin and Kudenko 2012).”
Learning by Playing - Solving Sparse Reward Tasks from Scratch (2018, background): “These include for example: shaping rewards (Ng et al., 1999; Randløv & Alstrøm, 1998; Gu et al., 2017), curriculum learning (Heess et al., 2017; Ghosh et al., 2018; Forestier et al., 2017), transfer of learned policies from simulation to reality (see Duan et al. (2017); Sadeghi et al. (2017); Tobin…”
Reinforcement Learning: An Introduction (1998, background): “Other examples of shaping in reinforcement learning are Gullapalli and Barto (1992), Mahadevan and Connell (1992), Mataric (1994), Dorigo and Colombette (1994), Saksida, Raymond, and Touretzky (1997), and Randløv and Alstrøm (1998).”
Temporal-Logic-Based Reward Shaping for Continuing Reinforcement Learning Tasks (2020, background): “(Randløv and Alstrøm 1998) added an intuitive signal to the reward of an agent learning to ride a bicycle, but had the unintended effect of causing the agent to ride in circles.”
Q( λ ) with Oﬀ-Policy Corrections (methodology): “To this end, we applied Q ∗ ( λ ) to the Bicycle domain [11].”
Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizations (2013, methodology): “Our HiP-MDP model rapidly identifies dynamics and near-optimal control policies for novel instances of the acrobot [Sutton and Barto, 1998] and bicycle [Randlov and Alstrom, 1998] domains.”
Imitation Learning (2017, background): “…to perform and evaluate imitation learning, such as video games [Thurau et al. 2004a; Gorman 2009; Mnih et al. 2015; Geisler 2002; Ross and Bagnell 2010], AI simulators [Raza et al. 2012; Aler et al. 2005], and reinforcement learning benchmarks [Judah et al. 2012; Randlov and Alstrom 1998].”
Expressing Arbitrary Reward Functions as Potential-Based Advice (2015, background): “Mi is then the MDP 〈S,A, γi, T ,R + Fi〉.”
Reward (Mis)design for Autonomous Driving (2021, background): “The RL literature provides numerous catastrophic examples of such loopholes [40,2,35], which often involve learned trajectories that actually loop in physical space to repeatedly accrue some reward but prevent long-term progress (see Fig.”