Planning from Pixels using Inverse Dynamics Models
By Keiran Paster, Sheila A. McIlraith, Jimmy Ba. 2020
46 citations recorded by the source corpus.
Learning task-agnostic dynamics models in high-dimensional observation spaces can be challenging for model-based RL agents. We propose a novel way to learn latent world models by learning to predict sequences of future actions conditioned on task completion. These task-conditioned models adaptively focus modeling capacity on task-relevant dynamics, while simultaneously serving as an effective heuristic for planning with sparse rewards. We evaluate our method on challenging visual goal completion tasks and show a substantial increase in performance compared to prior model-free approaches.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Ruslan Salakhutdinov
- Igor Mordatch
- Yoshua Bengio
- Benjamin Eysenbach
- Trevor Darrell
In Computer Vision and Pattern Recognition
- Sergey Levine
- Trevor Darrell
- Luc Van Gool
- Dacheng Tao
- Abhinav Kumar Gupta
- Pieter Abbeel
- Jitendra Malik
- Li Fei-Fei
In Reinforcement Learning
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Benjamin Eysenbach
- Aravind Rajeswaran
- Igor Mordatch
- Ruslan Salakhutdinov
- Michael Janner
Citations in the corpus
- Decision Transformer: Reinforcement Learning via Sequence Modeling (2021, methodology): “[43] further use an LSTM with state goal conditioning for goal-conditoned online RL settings.”
- Offline Reinforcement Learning as One Big Sequence Modeling Problem (2021, methodology): “In the context of model-free RL, our method also resembles recently proposed work on goal relabeling (Andrychowicz et al., 2017; Rauber et al., 2019; Ghosh et al., 2021; Paster et al., 2021) and reward conditioning (Schmidhuber, 2019; Srivastava et al.”
- Contrastive Learning as Goal-Conditioned Reinforcement Learning (2022, background): “Note that setting λ = 1 corresponds to GCBC [15, 21, 24, 40, 83, 96, 116, 119], which we will include as a baseline.”
- You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environments (2022, unknown): “[19] proposed to predict multi-step action sequences, showing a connection between sequence modeling and optimal open-loop planning.”
- Learning Domain Invariant Representations in Goal-conditioned Block MDPs (2021, background): “We believe the perfect alignment criteria will enable applications in diverse problem settings and offers interesting directions for future work, such as extensions to other goal-conditioned learning frameworks [9, 10].”