Backpropagation and Biological Plausibility
By Alessandro Betti, Marco Gori, Giuseppe Marra. 2018
5 citations recorded by the source corpus.
By and large, Backpropagation (BP) is regarded as one of the most important neural computation algorithms at the basis of the progress in machine learning, including the recent advances in deep learning. However, its computational structure has been the source of many debates on its arguable biological plausibility. In this paper, it is shown that when framing supervised learning in the Lagrangian framework, while one can see a natural emergence of Backpropagation, biologically plausible local algorithms can also be devised that are based on the search for saddle points in the learning adjoint space composed of weights, neural outputs, and Lagrangian multipliers. This might open the doors to a truly novel class of learning algorithms where, because of the introduction of the notion of support neurons, the optimization scheme also plays a fundamental role in the construction of the architecture.
Subfields: Artificial Intelligence, Cognitive Neuroscience, Optimization & Training, Neuroscience
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Simone Scardapane
- Michele Scarpiniti
- Aurelio Uncini
- Enzo Baccarelli
- Jiangpeng He
- Feng Zhu
- Laura Palagi
In Cognitive Neuroscience
In Optimization & Training
Citations in the corpus
- Why Should We Add Early Exits to Neural Networks? (2020, background): “In fact, backpropagation in classical neural networks is generally considered to be biologically implausible for multiple reasons [ 11 ] , i.e.,: (i) the need to store the outputs for each intermediate layer f i , to re-use it in the backward pass; (ii) the symmetry of weight matrices between…”