Towards a Biologically Plausible Backprop

By Benjamin Scellier, Yoshua Bengio. 2016

43 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Toward an Integration of Deep Learning and Neuroscience (2016, methodology): “This dynamics then allows the STDP-based rule to correspond to gradient descent on the energy function with respect to the weights (Scellier and Bengio, 2016).”
Towards deep learning with segregated dendrites (2016, methodology): “But they depend on feedback synapses that are symmetric with feedforward synapses to solve the credit assignment problem (Scellier and Bengio, 2016; Hinton et al., 2006), which is essentially a version of weight transport.”
Backpropagation and the brain (2020, background): “…error vectors locally within a layer, rather than transmitting them across layers, as in backprop and feedback alignment 57 , many algorithms in this class can be shown to make updates that approximately follow (and in some cases, exactly follow) the gradient computed by backprop 39,133,135,136 .”
Backpropagation and Biological Plausibility (2018, background): “Since then, a number of papers have contributed with many insights on the overcoming of this limitation, including recent contributions [1],[5].”
Reward-based training of recurrent neural networks for cognitive and value-based tasks (2016, background): “…in ’spiking’ stochastic units ( Lillicrap et al., 2016 ), and that approximate forms of backpropagation and SGD can be implemented in a biologically plausible manner ( Scellier and Bengio, 2016 ), including both spatially and temporally asynchronous updates in RNNs ( Jaderberg et al., 2016 ).”
Sparse Attentive Backtracking: Temporal CreditAssignment Through Reminding (2018, background): “Although some progress has been made toward credit-assignment mechanisms that are functionally equivalent to back-propagation (Lee et al., 2014; Scellier & Bengio, 2016; Whittington & Bogacz, 2017), it remains very unclear how the equivalent of back-propagation through time, used to train recurrent…”
Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits (2020, background): “…steering [16,34], multiplexing [1,45,51,69], alignment [2,17,18,59] or linearity [67] in isolation, often by learning in an offline fashion [2,4,40,47,59,83,84,101], without learning rules based on spikes [4,53,80,82–84] or without learning to solve tasks that necessitate hierarchical processing.”
An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity (2017, background): “…the second class explicitly approximate the backpropagation algorithm (O’Reilly, 1998; Lillicrap, Cownden, Tweed, & Akerman, 2016; Balduzzi, Vanchinathan, & Buhmann, 2014; Bengio, 2014; Bengio, Lee, Bornschein, & Lin, 2015; Scellier & Bengio, 2016), and we will compare them in detail in section 4.”