Meta-learning by the Baldwin effect

By Chrisantha Fernando, Jakub Sygnowski, Simon Osindero, Jane X. Wang, Tom Schaul, Denis Teplyashin, Pablo Sprechmann, Alexander Pritzel, Andrei A. Rusu. 2018

42 citations recorded by the source corpus.

The scope of the Baldwin effect was recently called into question by two papers that closely examined the seminal work of Hinton and Nowlan. To this date there has been no demonstration of its necessity in empirically challenging tasks. Here we show that the Baldwin effect is capable of evolving few-shot supervised and reinforcement learning mechanisms, by shaping the hyperparameters and the initial parameters of deep learning algorithms. Furthermore it can genetically accommodate strong learning biases on the same set of problems as a recent machine learning algorithm called MAML "Model Agnostic Meta-Learning" which uses second-order gradients instead of evolution to learn a set of reference parameters (initial weights) that can allow rapid adaptation to tasks sampled from a distribution. Whilst in simple cases MAML is more data efficient than the Baldwin effect, the Baldwin effect is more general in that it does not require gradients to be backpropagated to the reference parameters or hyperparameters, and permits effectively any number of gradient updates in the inner loop. The Baldwin effect learns strong learning dependent biases, rather than purely genetically accommodating fixed behaviours in a learning independent manner.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning (2019, methodology): “Meta-reinforcement learning methods have been evaluated on a number of different problems, including maze navigation [18, 19, 20], continuous control domains with parametric variation across tasks [10, 21, 22, 23], bandit problems [19, 18, 20, 24], levels of an arcade game [25], and locomotion tasks”
Meta-Learning in Neural Networks: A Survey (2020, background): “explored in RL including learning the initial conditions [19], [170], hyperparameters [170], [174], step directions [77] and step sizes [172].”
Reinforcement Learning, Fast and Slow. (2019, unknown): “Recent AI work has delved increasingly into methods for building agent architectures, as well as reward functions, through evolutionary algorithms, inspired by natural selection [15, 64, 65, 66, 67].”
Meta-learnt priors slow down catastrophic forgetting in neural networks (2019, background): “Thus the fitness / objective optimized by evolution is not the performance over a single task, nor only over a task distribution (as in previous meta-learning methods or Baldwinian meta-learning (Fernando et al., 2018)), but rather it correlates with the capacity of an agent or model to learn all ta”