DeepMDP: Learning Continuous Latent Space Models for Representation Learning

Citations in the corpus

Provable Representation Learning for Imitation with Contrastive Fourier Features (2021, methodology): “…to imitation learning, as they only provide guarantees bounding the performance difference between policies that are “close” ( e.g ., Lipschitz) in the representation space and say nothing regarding whether an arbitrary target policy in the true MDP can be represented in the latent MDP [20].”
Representation Matters: Offline Pretraining for Sequential Decision Making (2021, background): “Forward models of future representations – DeepMDP, Bisimulation – also exhibit poor performance.”
Contrastive Learning of Structured World Models (2019, background): “…the contrastive objective in Franc¸ois-Lavet et al. (2018), the mutual information objective in Anand et al. (2019), the distribution matching objective in Gelada et al. (2019) or using causality-based losses and physical priors in latent space (Jonschkowski & Brock, 2015; Ehrhardt et al., 2018).”
Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning (2020, background): “Other work that strives to learn a latent agent state representation include (DeepMDP Gelada et al., 2019) and (CRAR François-Lavet et al., 2019) that try to learn a transition model entirely in latent space.”
Offline Reinforcement Learning from Images with Latent Space Models (2020, methodology): “Prior works have tackled this problem with model-free RL methods by learning policies either from pixel inputs end-to-end (Haarnoja et al., 2018; Gelada et al., 2019; Singh et al., 2019; Kostrikov et al., 2020; Laskin et al., 2020; Han et al., 2020) or on top of unsupervised visual representations…”
Learning Markov State Abstractions for Deep Reinforcement Learning (2021, methodology): “Much recent work (Shelhamer et al., 2016; Pathak et al., 2017; Ha & Schmidhuber, 2018; Gelada et al., 2019; Yarats et al., 2019; Kaiser et al., 2020; Laskin et al., 2020a,b; Zhang et al., 2021) has sought to close this representation gap by incorporating a wide range of representation-learning…”
Learning Domain Invariant Representations in Goal-conditioned Block MDPs (2021, background): “Learning Invariants in RL: Robustness to domain shifts is crucial for real-world applications of RL. [6, 7, 50] implement the model-bisimulation framework [12] to acquire a minimal but sufﬁcient representation for solving the MDP problem.”
The Value Equivalence Principle for Model-Based Reinforcement Learning (2020, background): “This is an instantiation of the problem known as representation learning [43, 21, 9, 18, 45, 44, 19, 7].”
Predictive Information Accelerates Learning in RL (2020, methodology): “This work falls in another group, where future prediction is used as an auxiliary or representation learning method for model-free RL agents [1, 10, 14, 22, 29, 30, 33, 34, 35, 37].”