Direct-fit to nature: an evolutionary perspective on biological (and artificial) neural networks

By Uri Hasson, Samuel A. Nastase, Ariel Goldstein. bioRxiv, 2019

343 citations recorded by the source corpus.

Evolution is a blind fitting process by which organisms, over generations, adapt to the niches of an ever-changing environment. Does the mammalian brain use similar brute-force fitting processes to learn how to perceive and act upon the world? Recent advances in training deep neural networks has exposed the power of optimizing millions of synaptic weights to map millions of observations along ecologically relevant objective functions. This class of models has dramatically outstripped simpler, more intuitive models, operating robustly in real-life contexts spanning perception, language, and action coordination. These models do not learn an explicit, human-interpretable representation of the underlying structure of the data; rather, they use local computations to interpolate over task-relevant manifolds in a high-dimensional parameter space. Furthermore, counterintuitively, over-parameterized models, similarly to evolutionary processes, can be simple and parsimonious as they provide a versatile, robust solution for learning a diverse set of functions. In contrast to traditional scientific models, where the ultimate goal is interpretability, over-parameterized models eschew interpretability in favor of solving real-life problems or tasks. We contend that over-parameterized blind fitting presents a radical challenge to many of the underlying assumptions and practices in computational neuroscience and cognitive psychology. At the same time, this shift in perspective informs longstanding debates and establishes unexpected links with evolution, ecological psychology, and artificial life.

Subfields: Cognitive Neuroscience, Cultural Studies

View Paper: https://www.semanticscholar.org/paper/8b127b9c7e37f075559ed5a51428527205dcf3f5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Cultural Studies

Citations in the corpus

Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future (2020, background): “something has forced a reshaping of the questions around the study of the visual system. Neuroscientists are adapting to this new style of explanation and the different expectations that come with it [121]. Importantly, these models have also made it possible to reach some of the pre-existing goals ”
Shared computational principles for language processing in humans and deep language models (2022, background): “In contrast, DLMs aim to provide the appropriate linguistic output given the prior statistics of language use in similar contexts 20,59 .”
A connectome of the Drosophila central complex reveals network motifs suitable for flexible navigation and context-dependent action selection (2020, background): “…learning, a broader question concerns the level of understanding that is appropriate or even possible for the function of large and complex neural networks (Gao and Ganguli, 2015; Hasson et al., 2020; Lillicrap and Kording, 2019; Richards et al., 2019; Saxe et al., 2020; Yamins and DiCarlo, 2016).”
Descriptive, Predictive and Explanatory Personality Research: Different Goals, Different Approaches, but a Shared Need to Move beyond the Big Few Traits (2020, background): “Many of these models capitalize on so many parameters and small variations in them that they may never be fathomable by humans: not because the models are overly complex per se , but because human minds have constraints that models do not have to obey (Hasson, Nastase, & Goldstein, 2020).”
If deep learning is the answer, what is the question? (2020, background): “A related proposal draws an analogy between optimization over computation in neural networks and optimization over biological forms by evolution: in both cases, interpretable functional adaptations emerge without meaningful constraints being imposed on the search process 32 .”
Neuroevolution of self-interpretable agents (2020, unknown): “This constraint naturally incorporates into our agent an inductive bias that determines what it does well at [29, 41, 102], and this bias is dependent on the choice of our indirect encoding method.”
Evaluating large language models in theory of mind tasks (2023, background): “While the Chinese room argument became widely popular, many scholars believe it is flawed, especially in the context of contemporary connectionist AI systems like AlphaZero or LLMs ( 69 – 72 ).”