Integrative Benchmarking to Advance Neurally Mechanistic Models of Human Intelligence.

By Martin Schrimpf, Jonas Kubilius, Michael J. Lee, N. Murty, James J. DiCarlo. Neuron, 2020

298 citations recorded by the source corpus.

A potentially organizing goal of the brain and cognitive sciences is to accurately explain domains of human intelligence as executable, neurally mechanistic models. Years of research have led to models that capture experimental results in individual behavioral tasks and individual brain regions. We here advocate for taking the next step: integrating experimental results from many laboratories into suites of benchmarks that, when considered together, push mechanistic models toward explaining entire domains of intelligence, such as vision, language, and motor control. Given recent successes of neurally mechanistic models and the surging availability of neural, anatomical, and behavioral data, we believe that now is the time to create integrative benchmarking platforms that incentivize ambitious, unified models. This perspective discusses the advantages and the challenges of this approach and proposes specific steps to achieve this goal in the domain of visual intelligence with the case study of an integrative benchmarking platform called Brain-Score.

Subfields: Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/3ecc7ac21c22af073f37294df341c51e5d2d576d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

The neural architecture of language: Integrative modeling converges on predictive processing (2020, background): “…transformed our algorithmic understanding of the primate ventral visual stream (Bao et al., 2020; Cadena et al., 2019; Cichy et al., 2016; Kietzmann et al., 2019; Kubilius et al., 2019; Schrimpf et al., 2018, 2020; Yamins et al., 2014), and holds great promise for other aspects of brain function.”
Neuromorphic computing at scale (2025, unknown): “There are also tools to evaluate the performance of neuromorphic systems and models for neuroscience and psychological modelling 95,96 .”
Sparse Prototype Code Underlies Classification and Prediction Across Modalities (2026, unknown): “In parallel, deep networks have been shown to predict responses in primate ventral-stream areas, including IT cortex [Yamins et al., 2014, Schrimpf et al., 2020], and object-manifold theory has been applied to biological populations [Froudarakis et al., 2020, Sorscher et al., 2022, Kuoch et al.,…”
Targeted perturbations reveal brain-like local coding axes in robustified, but not standard, ANN-based brain models (2025, unknown): “This analysis replicates prior results and highlights a key challenge in the field: predictive accuracy alone does not meaningfully distinguish between candidate ANN-based models (Canatar et al., 2023; Tuckute et al., 2022; Conwell et al., 2023; Schrimpf et al., 2020; Ratan Murty et al., 2021).”