Biologically Plausible Error-Driven Learning Using Local Activation Differences: The Generalized Recirculation Algorithm

By Randall C. O’Reilly. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep Networks with Internal Selective Attention through Feedback Connections (2014, background): “In the context of computer vision, RL has been shown to be able to learn saccades in visual scenes to learn selective attention [40, 41], learn feedback to lower levels [42, 43], and improve face recognition [44, 45].”
Toward an Integration of Deep Learning and Neuroscience (2016, methodology): “These include Generalized Recirculation (O’Reilly, 1996), Contrastive Hebbian Learning (Xie and Seung, 2003), random feedback weights together with synaptic homeostasis (Lillicrap et al., 2014; Liao et al., 2015), spike timing dependent plasticity (STDP) with iterative inference and target propagati”
Theory of cortical function (2017, background): “Max pooling (also called softmax) transmits the most active response among a set of inputs (72).”
Random feedback weights support learning in deep neural networks (2014, methodology): “More-plausible algorithms have been devised 9,10,13,14,22,26,27,28,29,30 , but these either fall far short of backprop’s speed or call for a lot of additional processing 10,11,12,13,22 .”
Theories of Error Back-Propagation in the Brain (2019, background): “O ’ Reilly [37] demonstrated that in the presence of backward connections, the information about the target pattern propagates to earlier layers, and an analogous sequence of weight modi ﬁ cations in the hidden layers also approximates a version of the back-propagation algorithm for recurrent…”
How Important Is Weight Symmetry in Backpropagation? (2015, background): “[O’Reilly 1996] O’Reilly, R. C. 1996.”
An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity (2017, methodology): “The models in the second class explicitly approximate the backpropagation algorithm (O’Reilly, 1998; Lillicrap, Cownden, Tweed, & Akerman, 2016; Balduzzi, Vanchinathan, & Buhmann, 2014; Bengio, 2014; Bengio, Lee, Bornschein, & Lin, 2015; Scellier & Bengio, 2016), and we will compare them in detail…”
Switching between internal and external modes: A multiscale learning principle (2017, background): “& Williams, 1986), or individual units within each layer of a network can compute mismatch based on more local information (e.g., Bengio, Lee, Bornschein, Mesnard, & Lin, 2015; Hinton & McClelland, 1988; O’Reilly, 1996; Smolensky, 1986).”
Equilibrium Propagation: Bridging the Gap between Energy-Based Models and Backpropagation (2016, background): “Note that algorithms related to ours based on infinitesimal perturbations at the outputs were also proposed by O’Reilly (1996); Hertz et al. (1997).”
Connectionist modelling in psychology: A localist manifesto (2000, methodology): “As an example Plaut & McClelland cite O’Reilly’s (1996) GeneRec algorithm that avoids the use of nonlocal information in its implementation of gradient descent by running its network in two phases and by mirroring forward going connections with backward going connections of (ideally) the same weight”
Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures (2018, background): “Researchers have long sought biologically plausible and empirically powerful learning algorithms that avoid these flaws [2, 30, 31, 1, 26, 39, 14, 16, 12, 5, 23].”