Dendritic cortical microcircuits approximate the backpropagation algorithm

By João Sacramento, Rui Ponte Costa, Yoshua Bengio, Walter Senn. Neural Information Processing Systems, 2018

373 citations recorded by the source corpus.

Deep learning has seen remarkable developments over the last years, many of them inspired by neuroscience. However, the main learning mechanism behind these advances - error backpropagation - appears to be at odds with neurobiology. Here, we introduce a multilayer neuronal network model with simplified dendritic compartments in which error-driven synaptic plasticity adapts the network towards a global desired output. In contrast to previous work our model does not require separate phases and synaptic learning is driven by local dendritic prediction errors continuously in time. Such errors originate at apical dendrites and occur due to a mismatch between predictive input from lateral interneurons and activity from actual top-down feedback. Through the use of simple dendritic compartments and different cell-types our model can represent both error and normal activity within a pyramidal neuron. We demonstrate the learning capabilities of the model in regression and classification tasks, and show analytically that it approximates the error backpropagation algorithm. Moreover, our framework is consistent with recent observations of learning between brain areas and the architecture of cortical microcircuits. Overall, we introduce a novel view of learning on dendritic cortical circuits and on how the brain may solve the long-standing synaptic credit assignment problem.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/4a684d8ff381be9334195e850d09b2a027447230

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Electrical and Electronic Engineering

Citations in the corpus

Artificial Neural Networks for Neuroscientists: A Primer (2020, methodology): “To implement predictive coding in ANNs, feedback connections from higher layers can be trained with a separate loss that compares the output of feedback connections with the neural activity in lower layers (Lotter et al., 2016; Sacramento et al., 2018).”
Synaptic Plasticity Forms and Functions. (2020, background): “…of network innervate the distal dendritic regions in relative isolation from the feedforward input, allow error-like signals to be locally calculated within the dendrites of individual hidden-layer neurons (Körding & König 2001, Urbanczik & Senn 2014, Guerguiev et al. 2017, Sacramento et al. 2018).”
Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits (2020, result): “S5), including distal apical 300 inhibition [30,59], which is consistent with the action of somatostatin-positive Martinotti cells in cortical 301 circuits [45, 62].”
Training Neural Networks with Local Error Signals (2019, methodology): “An online version of feedback alignment learning can be implemented in a biologically plausible way using multi-compartment neurons (Guerguiev et al., 2017), and in cortical microcir-cuits for continuous learning without separate forward and backward passes (Sacramento et al., 2018).”
Theories of Error Back-Propagation in the Brain (2019, methodology): “In this review article, we focus on a popular model called the dendritic error model [13].”
Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future (2020, background): “That being said, several researchers are investigating means by which the brain could perform something like backpropagation[111, 112, 113, 114].”