Learning to Adapt: Meta-Learning for Model-Based Control

By Ignasi Clavera, Anusha Nagabandi, Ronald S. Fearing, Pieter Abbeel, Sergey Levine, Chelsea Finn. 2018

94 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

Citations in the corpus

Solving Rubik's Cube with a Robot Hand (2019, background): “A common theme in research is to try to exploit a shared structure in a distribution of environments, to quickly identify and adapt to previously unseen cases [55, 94, 47, 54] There are works that directly treat meta-learning as identification of a dynamics model of the environment [19] while others”
Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks (2022, background): “, robotics control [51], neural translation [52] and uncertainty estimation [53], and many improvements [54], [55], [56], [57], [58] have been made to boost its performance.”
CoMPS: Continual Meta Policy Search (2021, background): “, 2019; Dorfman & Tamar, 2020), applying gradient descent from initial parameters (Finn et al., 2017; Gupta et al., 2018b; Rothfuss et al., 2018; Clavera et al., 2018; Mendonca et al., 2019; Zintgraf et al., 2019; Mitchell et al., 2020), and training a critic to provide policy gradients (Sung et al.”
RoboNet: Large-Scale Multi-Robot Learning (2019, background): “…34], transfer across different dynamics [35, 36, 37, 2], transfer across robot morphologies with invariant feature spaces [38] and modularity [39], transfer across viewpoints through recurrent control [40], and transfer across objects [41, 42], tasks [43] or environments [44] through meta-learning.”
Model-Based Reinforcement Learning via Meta-Policy Optimization (2018, background): “Several works approach this problem of model-bias by learning a distribution of models [7], [10], [23], [26], or by learning adaptive models [27]–[29].”
Intrinsic Motivation for Encouraging Synergistic Behavior (2020, background): “However, recent successful applications of model-based reinforcement learning (Nagabandi et al., 2018; Clavera et al., 2019) have used deterministic regressors, modeling just the maximum likelihood transitions.”
Guiding Policies with Language via Meta-Learning (2018, background): “, 2017), model-based RL (Clavera et al., 2018), a wide range of supervised tasks (Snell et al.”
Assessing Generalization in Deep Reinforcement Learning (2018, methodology): “…problem and is under active investigation (Wang et al., 2016; Duan et al., 2016b; Rajeswaran et al., 2017a; Pinto et al., 2017; Finn et al., 2017; Kansky et al., 2017; Yu et al., 2017; Sung et al., 2017; Leike et al., 2017; Al-Shedivat et al., 2018; Clavera et al., 2018; Sæmundsson et al., 2018).”
Continual Model-Based Reinforcement Learning with Hypernetworks (2020, unknown): “Relationship to Meta-Learning Our work is different from meta-learning approaches in MBRL like [27], [28] in two ways.”
Edge-Labeling Graph Neural Network for Few-Shot Learning (2019, background): “A lot of interest in meta-learning [1] has been recently arisen in various areas including especially taskgeneralization problems such as few-shot learning [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], learn-to-learn [16, 17, 18], non-stationary reinforcement learning[19, 20, 21], and continual ”
Few-shot learning with adaptively initialized task optimizer: a practical meta-learning approach (2019, unknown): “MAMLhavebeenappliedinvariousscenarios,such asuncertaintyestimation(Finnetal.2018),roboticscontrol(Yuetal.2018;Claveraetal.”
Learning to Infer and Execute 3D Shape Programs (2019, methodology): “The very recent SPIRAL system (Clavera et al., 2018) also takes as its goal to learn structured program–like models from (2D) images.”
How to Train Your MAML to Excel in Few-Shot Classification (2021, methodology): “MAML has been applied in various scenarios, such as uncertainty estimation [32], robotics control [16, 113], neural translation [37], language generation [42], etc.”
Small Sample Learning in Big Data Era (2018, methodology): “Unlike model-free RL (Wang et al., 2016; Duan et al., 2016; Sung et al., 2018; Al-Shedivat et al., 2018), Clavera et al. (2018) considered to learn online adaptation in the context of model-based reinforcement learning.”
Guided Meta-Policy Search (2019, unknown): “Prior methods learned reinforcement learners represented by a recurrent or recursive neural network [7, 48, 24, 41, 33], using gradient descent from a learned initialization [8, 14, 36], using a learned critic that provides gradients to the policy [44, 17], or using a planner and an adaptable model ”
Generalized Hidden Parameter MDPs Transferable Model-based RL in a Handful of Trials (2020, methodology): “Because reward varies across tasks in our experiment, we do not compare against adaptive model-based methods like (Clavera et al. 2018) that do not model reward.”
Deep Reinforcement Learning for Closed-Loop Blood Glucose Control (2020, background): “RL is a promising solution, as it is well-suited to learning complex behavior and readily adapts to changing domains (Clavera et al., 2018).”
Meta Reinforcement Learning with Task Embedding and Shared Policy (2019, methodology): “, 2018], the environment model for model-based RL [Clavera et al., 2018], and the reward functions in the context of unsupervised learning and inverse RL respectively [Gupta et al.”