Learning to Learn for Global Optimization of Black Box Functions
By Yutian Chen, Matthew W. Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Timothy P. Lillicrap, Nando de Freitas. 2016
43 citations recorded by the source corpus.
We present a learning to learn approach for training recurrent neural networks to perform black-box global optimization. In the meta-learning phase we use a large set of smooth target functions to learn a recurrent neural network (RNN) optimizer, which is either a long-short term memory network or a differentiable neural computer. After learning, the RNN can be applied to learn policies in reinforcement learning, as well as other black-box learning tasks, including continuous correlated bandits and experimental design. We compare this approach to Bayesian optimization, with emphasis on the issues of computation speed, horizon length, and exploration-exploitation trade-offs.
Subfields: Artificial Intelligence, Education, Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Pieter Abbeel
- Yoshua Bengio
- Quoc V. Le
- Ruslan Salakhutdinov
- Chelsea Finn
- Mohit Bansal
- Jianfeng Gao
In Education
In Optimization & Training
- Quoc V. Le
- Razvan Pascanu
- Hado van Hasselt
- Irwan Bello
- Jascha Narain Sohl-Dickstein
- Yoshua Bengio
- David Silver
- Sebastian Flennerhag
Citations in the corpus
- Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions (2017, methodology): “Chen et al. (2017) showed how to learn to learn by gradient descent in the black-box optimization setting.”
- Learning Combinatorial Optimization Algorithms over Graphs (2017, methodology): “[8] Chen, Yutian, Hoffman, Matthew W, Colmenarejo, Sergio Gomez, Denil, Misha, Lillicrap, Timothy P, and de Freitas, Nando.”
- Neural Combinatorial Optimization with Reinforcement Learning (2016, methodology): “Motivated by the recent advancements in sequence-to-sequence learning (Sutskever et al., 2014), neural networks are again the subject of study for optimization in various domains (Yutian et al., 2016), including discrete ones (Zoph & Le, 2016).”
- Learned Optimizers that Scale and Generalize (2017, background): “Finally, Chen et al. (2016) also learn reinforcement learning, but by supervised meta-training of the meta-learner.”
- Discovery of Useful Questions as Auxiliary Tasks (2019, background): “…et al., 2018), unsupervised learning (Gupta et al., 2018; Hsu et al., 2018), few-shot model adaptation (Nagabandi et al., 2018), and improving the optimizers (Andrychowicz et al., 2016; Li & Malik, 2017; Ravi & Larochelle, 2017; Wichrowska et al., 2017; Chen et al., 2016; Gupta et al., 2018).”
- Learning Gradient Descent: Better Generalization and Longer Horizons (2017, methodology): “In another recent paper (Chen et al., 2016), RNN is used to take current position and value as input and output next position, and it works well for black-box optimization and simple RL tasks.”
- Bootstrapped Meta-Learning (2021, methodology): “Similarly, optimizing the learner’s performance after K updates can fail to account for the process of learning, causing another form of myopia (Flennerhag et al., 2019; Stadie et al., 2018; Chen et al., 2016; Cao et al., 2019).”
- Meta-learning of Sequential Strategies (2019, methodology): “…learning process, such as learning the optimizer [Andrychowicz et al., 2016, Li and Malik, 2016, Ravi and Larochelle, 2016, Wichrowska et al., 2017, Chen et al., 2018b], the metric space [Vinyals et al., 2016, Snell et al., 2017], the initial network parameters [Finn et al., 2017, Nichol and…”