Learning to Achieve Goals

By Leslie Pack Kaelbling. 1993

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog (2019, neutral): “[30, 2]).”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “The earliest work on goal-based RL dates back to (Kaelbling, 1993).”
Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning (2019, background): “Our work builds on goal-conditioned RL [17, 18, 19, 20], which has been explored in the context of reward-free learning [21], learning with sparse rewards [18], large scale generalizable imitation learning [22], and hierarchical RL [15].”
Visual Reinforcement Learning with Imagined Goals (2018, methodology): “ on real-world robotic systems, without access to ground-truth state or reward information during training. Our method uses a goal-conditioned value function [40] in order to solve more general tasks [45, 18]. To improve the sample-efﬁciency of our method during off-policy training, we retroactively”
Learning Latent Plans from Play (2019, background): “Goal conditioned policies have been explored extensively in the literature for reinforcement learning [23, 44, 36, 52, 3, 32, 48, 8, 58], as well as for control via inverse models [1, 35, 9, 61].”
Reward-Conditioned Policies (2019, methodology): “The central idea behind our method – that suboptimal trajectories can serve as optimal supervision for other tasks or problems – has recently been explored for goal -conditioned policies, both with reinforcement learning [1, 16, 34] and supervised learning [8].”
Training Agents using Upside-Down Reinforcement Learning (2019, methodology): “O -policy training can be augmented with goal-conditional value functions [11] (see also Pong et al.”
Search on the Replay Buffer: Bridging Planning and Reinforcement Learning (2019, methodology): “One class of methods (Bacon et al., 2017; Frans et al., 2017; Kaelbling, 1993a; Kulkarni et al., 2016; Nachum et al., 2018; Parr and Russell, 1998; Precup, 2000; Sutton et al., 1999; Vezhnevets et al., 2017) jointly learn a low-level policy for performing each of the skills together with a…”
Multi-timescale nexting in a reinforcement learning robot (2011, methodology): “Note that this technique is applicable even when experience is gathered from the physical world, where no formal notion of state is available.”
Q-Learning in enormous action spaces via amortized approximate maximization (2020, methodology): “Q-learning can also be easily and robustly combined with exotic exploration strategies (Ostrovski et al., 2017; Mnih et al., 2016; Horgan et al., 2018) as well as data augmentation through post-hoc modiﬁcation (Kaelbling, 1993; Andrychowicz et al., 2017).”
If MaxEnt RL is the Answer, What is the Question? (2019, background): “While prior work on goal-reaching [3, 22, 43] assumes that the goal state is observed, the goal-reaching meta-POMDP only assumes that the policy has a belief about the goal state.”
Planning to Explore via Self-Supervised World Models (2020, background): “Model-based control Model-free agents are often data-inefﬁcient (Kaelbling et al., 1996) and hard to adapt to different tasks, although one promising avenue for adapting these agents is goal-conditioned reinforcement learning (Kaelbling, 1993; Pathak et al., 2018; Pong et al., 2019).”
Learning to Learn (1998, unknown): “Transfer methods have been applied to neural networks, and neural networks have been used to model transfer processes in cognition (see Holyoak & Barnden (1994), and the review by Pratt & Jennings (1996».”
CURIOUS: Intrinsically Motivated Multi-Task, Multi-Goal Reinforcement Learning (2018, background): “When the set of goals/tasks is finite [9; 4], each transition is generally used to update the policy on every other goal/task respectively.”
Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation (2021, methodology): “While many approaches have been taken to task-specification, including task IDs [54, 55], robot and human demonstrations [56, 57, 58], and meta-learning from rewards [59], a common approach is goalconditioned learning [60, 61, 2, 1], where an agent learns to reach particular goal states or distribut”
Skew-Fit: State-Covering Self-Supervised Reinforcement Learning (2019, methodology): “While most goal-conditioned RL methods (Kaelbling, 1993; Lillicrap et al., 2016; Schaul et al., 2015; Andrychowicz et al., 2017; Nair et al., 2018; Pong et al., 2018; Florensa et al., 2018a).”
γ-Models: Generative Temporal Difference Learning for Infinite-Horizon Prediction (2020, unknown): “TDMs interpret models as a special case of goal-conditioned value functions (Kaelbling, 1993; Foster & Dayan, 2002; Schaul et al., 2015; Andrychowicz et al., 2017), though the TDM is constrained to predict at a fixed horizon and is limited to tasks for which the reward depends only on the last state”
Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewards (2019, background): “Tasks with well-defined goals provide an interesting extension of the traditional RL framework (Kaelbling, 1993; Sutton et al., 2011; Schaul et al., 2015).”