A critique of pure learning and what artificial neural networks can learn from animal brains

By Anthony M. Zador. 2019

547 citations recorded by the source corpus.

Artificial neural networks (ANNs) have undergone a revolution, catalyzed by better supervised learning algorithms. However, in stark contrast to young animals (including humans), training such networks requires enormous numbers of labeled examples, leading to the belief that animals must rely instead mainly on unsupervised learning. Here we argue that most animal behavior is not the result of clever learning algorithms—supervised or unsupervised—but is encoded in the genome. Specifically, animals are born with highly structured brain connectivity, which enables them to learn very rapidly. Because the wiring diagram is far too complex to be specified explicitly in the genome, it must be compressed through a “genomic bottleneck”. The genomic bottleneck suggests a path toward ANNs capable of rapid learning. Recent gains in artificial neural networks rely heavily on large amounts of training data. Here, the author suggests that for AI to learn from animal brains, it is important to consider that animal behaviour results from brain connectivity specified in the genome through evolution, and not due to unique learning algorithms.

Subfields: Artificial Intelligence, Cell Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cell Biology

Citations in the corpus

Weight Agnostic Neural Networks (2019, background): “A recent neuroscience commentary, “What artificial neural networks can learn from animal brains” [125] provides a critique of how learning (and also meta-learning) is currently implemented in artificial neural networks.”
The hardware lottery (2020, background): “In addition, our brains are able to learn efficient representations from far fewer labelled examples than deep neural networks (Zador, 2019).”
The neural architecture of language: Integrative modeling converges on predictive processing (2020, background): “Perhaps 230 most of development is then a combination of the system wiring up (Saygin et al., 2016; Zador, 2019) and learning the right 231 decoders on top of largely structurally defined features.”
Neuroevolution of self-interpretable agents (2020, background): “solution to a small subspace of all possible solutions offered by direct encoding methods. This constraint naturally incorporates into our agent an inductive bias that determines what it does well at [29, 42, 103], and this bias is dependent on the choice of our indirect encoding method. arXiv:2003.”
Continual Learning with Deep Artificial Neurons (2020, background): “Many in the machine learning community have begun to at least acknowledge that efficient learning in real brains may result, not from clever learning algorithms, but from innate priors which have arisen through evolution, and which are generally kept fixed during intra-life deployment [42].”
Unsupervised neural network models of the ventral visual stream (2020, background): “Our results can be taken as a statement about the inductive biases that are needed to build an effective high-performing but biologically plausible visual learning system (66), suggesting that some form of contrastive objective function might be implemented by primates as a key “bias” shaping real l”
Direct-fit to nature: an evolutionary perspective on biological (and artificial) neural networks (2019, background): “In fact, the genome may impose exceptionally strong priors on learning (Zador, 2019).”
Introducing Artificial Intelligence Training in Medical Education (2019, unknown): “In a recent paper, Zador [32] argued that we have much more to learn from animal brains to unravel this phenomenon.”
A connectome of the Drosophila central complex reveals network motifs suitable for flexible navigation and context-dependent action selection (2020, background): “It is possible that developmentally-driven organizational features of natural brains may actually make them more tractable than artificial neural networks for an understanding of their function (Zador, 2019).”
If deep learning is the answer, what is the question? (2020, background): “The recent literature is studded with comparisons of the behaviour and activity of biological and artificial systems 7–21 , summarized in a growing number of review articles 22–30 .”
Engineering a Less Artificial Intelligence. (2019, background): “, 2011) and generations of ancestors that were selected through evolutionary pressure to have a good architecture in that environment (Zador, 2019).”
Robot Programming without Coding (2020, methodology): “As [16] points out recently, looking at infants and comparing their learning process to what state of the art Artificial Neural Networks (ANNs) can do in tasks as e.”
Predicting the Future from First Person (Egocentric) Vision: A Survey (2021, background): “From the psychological and physiological research [155], it is clear that learning end-to-end from labels is not the only way the human brain works.”