Learning a synaptic learning rule

By Yoshua Bengio, Samy Bengio, Jocelyn Cloutier. 1991

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Meta & Continual Learning

Citations in the corpus

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (2017, background): “A popular approach for meta-learning is to train a meta-learner that learns how to up-date the parameters of the learner’s model (Bengio et al., 1992; Schmidhuber, 1992; Bengio et al., 1990).”
Learning to Optimize Neural Nets (2017, background): “Closely related is (Bengio et al., 1991), which learns a Hebb-like synaptic learning rule that does not depend on the objective function, which does not allow for generalization to different objective functions.”
CrossTransformers: spatially-aware few-shot transfer (2020, background): “Few-shot learning [25, 35, 46, 53] has recently been primarily addressed in the meta-learning framework [57, 71, 80], where a model learns an update rule for the parameters of a base-learner model [6, 7, 72] through a sequence of training episodes [79, 88].”
Learning to Learn with Generative Models of Neural Network Checkpoints (2022, background): “Of course, the concept of learning improved optimizers is not new and dates back to the 1980s, if not earlier, following early work from Schmidhuber (1987) and Bengio et al. (1991).”
Toward Causal Representation Learning (2021, background): “The contexts can likewise be interpreted as different tasks, which provides a connection to metalearning [23], [68], [213].”
N-BEATS: Neural basis expansion analysis for interpretable time series forecasting (2019, background): “The inner learning procedure is parameterized, conditioned or otherwise influenced by the outer learning procedure [3].”
Meta-SGD: Learning to Learn Quickly for Few Shot Learning (2017, background): “Meta-learning offers a new perspective to machine learning, by lifting the learning level from data to tasks [3, 21, 23].”
Meta Networks (2017, background): “The base and the meta-level models can be framed in a single learner (Schmidhuber, 1993) or in separate learners (Bengio et al., 1990; Hochreiter et al., 2001).”
Recasting Gradient-Based Meta-Learning as Hierarchical Bayes (2018, background): “This capacity for meta-learning also has the potential to enable an artiﬁcially intelligent agent to learn more efﬁciently in situations with little available data or limited computational resources (Schmidhuber, 1987; Bengio et al., 1991; Naik & Mammone, 1992).”
Bayesian Model-Agnostic Meta-Learning (2018, background): “This ability, also known as learning to learn or meta-learning (Biggs, 1985; Bengio et al., 1990), has recently obtained much attention in machine learning by formulating it as few-shot learning (Lake et al., 2015; Vinyals et al., 2016).”
Learning Rapid-Temporal Adaptations (2017, background): “The base and meta-level models can be framed as a single learner [34] or as separate learners [3, 14].”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, background): “Bengio et al. (1990, 1992) proposed the optimisation of a learning rule via evolutionary search.”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, methodology): “As depicted in the left diagram, the recursive process of learning to improve an agent’s own learning is decomposed into two separate learning processes called the inner loop and outer loop (Bengio et al., 1992; Bengio, Bengio, and Cloutier, 1990; Schmidhuber, 1992).”
Learning to Learn for Global Optimization of Black Box Functions (2016, methodology): “A series of papers from Bengio et al. [1990, 1992, 1995] present methods for local update rules that avoid back-propogation by using simple parametric rules, and Runarsson and Jonsson [2000] extend this to more complex update models.”
Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments (2017, background): “There is a wealth of work on meta-learning, including methods for learning update rules for neural models that were explored in the past [32–34], and more recent approaches that focused on learning optimizers for deep networks [13, 35–37], generating model parameters [38– 40], learning task…”
Modular meta-learning (2018, background): “Others [8, 9, 10, 11, 12] perform different types of parametric changes in the network’s computation conditioned on few ex-Preprint amples.”
Learned Optimizers that Scale and Generalize (2017, neutral): “A series of papers from Bengio et al. (1990; 1992; 1995) presents methods for learning parameterized local neural network update rules that avoid back-propagation.”