DARLA: Improving Zero-Shot Transfer in Reinforcement Learning

2017

Subfields: Reinforcement Learning, Representation & Self-Supervised Learning

Citations in the corpus

• Reconciling deep learning with symbolic artificial intelligence: representing objects and relations (2019, background): “The resulting representations are not only advantageous from the point of view of interpretability, but are also beneficial to downstream tasks such as reinforcement learning [34] and learning hierarchical concepts [35 ].”
• On the Transfer of Disentangled Representations in Realistic Settings (2020, background): “Disentangled representations hold the promise of generalization to unseen scenarios (Higgins et al., 2017b), increased interpretability (Adel et al., 2018; Higgins et al., 2018) and faster learning on downstream tasks (van Steenkiste et al., 2019; Locatello et al., 2019a).”
• Weakly-Supervised Disentanglement Without Compromises (2020, methodology): “…noise; we assume that noise is explicitly modeled as a latent variable and its effect is manifested through g? as done by (Bengio et al., 2013; Locatello et al., 2019b; Higgins et al., 2018; 2017a; Suter et al., 2019; Reed et al., 2015; LeCun et al., 2004; Kim & Mnih, 2018; Gondal et al., 2019).”
• Shaping Belief States with Generative Environment Models for RL (2019, background): “A non-exhaustive list includes pixel control [55], contrastive predictive coding (CPC) [56], action-conditional CPC [20], frame reconstruction [35, 57], next-frame prediction [18, 58, 20, 59] and successor representations [60, 61, 62].”
• On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset (2019, background): “sequence modeling, curiosity-based exploration or even domain adaptation in reinforcement learning [45, 25, 37, 16, 18, 29, 47].”
• On the Fairness of Disentangled Representations (2019, methodology): “Weaker forms of supervision like relational information or additional assumptions on the effect of the factors of variation were also studied [39, 16, 47, 31, 91, 26, 19, 41, 93, 62, 53, 81, 8] and applied in the sequential data and reinforcement learning settings [88, 85, 57, 69, 37, 38].”
• Are Disentangled Representations Helpful for Abstract Visual Reasoning? (2019, background): “Disentangled representations have been found to be more sample-efficient [29], less sensitive to nuisance variables [55], and better in terms of (systematic) generalization [1, 16, 28, 35, 70].”
• Multi-Object Representation Learning with Iterative Variational Inference (2019, methodology): “Recent breakthroughs in unsupervised representation learning (Higgins et al., 2017a; Makhzani et al., 2015; Chen et al., 2016) tend to focus on data where a single object of interest is placed in front of some background (e.g. dSprites, 3D Chairs, CelebA).”
• Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations (2018, methodology): “For this, we plot in Figure 4 (left) the FactorVAE score against different regularization strengths for each model on the Cars3D data set whereas Figure 16 in Appendix I.5 shows the same plot for all data sets and disentanglement metrics.”
• World Models (2018, methodology): “Our approach may complement sim2real approaches outlined in (Bousmalis et al., 2017; Higgins et al., 2017).”
• Building Generalizable Agents with a Realistic and Rich 3D Environment (2018, background): “Generalization: There is a recent trend in reinforcement learning focusing on the problem of generalization, ranging from learning to plan (Tamar et al., 2016), meta-learning (Duan et al., 2016; Finn et al., 2017a) to zero-shot learning (Andreas et al., 2016; Oh et al., 2017; Higgins et al., 2017).”
• A Deep Reinforcement Learning Perspective on Internet Congestion Control (2019, background): “While these empirical results are promising, the question of generalization, or transfer, in RL is an active field of research (Tamar et al., 2016; Barreto et al., 2017; Narasimhan et al., 2017; Higgins et al., 2017), and providing guarantees about the performance of a congestion control policy when”
• Transferring Knowledge across Learning Processes (2018, background): “Because such approaches can limit knowledge transfer (Higgins et al., 2017; Achille et al., 2018), applying standard transfer learning techniques leads to catastrophic forgetting , by which the model is rendered unable to perform a previously mastered task (McCloskey & Cohen, 1989; Goodfellow et…”
• Reinforcement Learning, Fast and Slow. (2019, background): “Possibilities suggested by AI research include constraints on the initial pattern of network connectivity [36]; shaping of synaptic learning rules [35, 68, 69]; and factors encouraging the emergence of disentangled or compositional representations [54, 70, 71] and internal predictive models [51].”
• Unsupervised Video Object Segmentation for Deep Reinforcement Learning (2018, methodology): “Previous works have used learned feature representations for reinforcement learning [7, 11, 10, 19].”
• Dynamics-aware Embeddings (2019, background): “Several papers have proposed using (variational) auto-encoders to learn embeddings for observations (Higgins et al., 2017; Caselles-Dupré et al., 2018); unlike our work, these models operate on a single observation at a time and do not depend on the environment dynamics.”
• Joint-VAE: Learning Disentangled Joint Continuous and Discrete Representations (2018, background): “Disentangled representations have also been applied to reinforcement learning (Higgins et al. (2017a)) and for learning visual concepts (Higgins et al. (2017b)).”
• Visual Semantic Navigation using Scene Priors (2018, background): “Harrison et al. (2017); Higgins et al. (2017) study zero-shot policy transfer in the scenarios that the dynamics or the states of the target domain is different from those of the source domain.”