The Measure of All Minds: Evaluating Natural and Artificial Intelligence

By JosÃ© Hernandez-Orallo. 2017

158 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Computational Theory and Mathematics, Evaluation & Benchmarks, Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Computational Theory and Mathematics

In Evaluation & Benchmarks

In Cognitive Science & Psychology

Citations in the corpus

The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation (2018, background): “, 2017 Although trends in performance across a range of domains have historically not been comprehensively tracked or well theorized (Brundage, 2016; Hernández-Orallo, 2017), there have been some recent efforts to track, measure, and compare performance (Eckersley and Nasser et al.”
On Defining Artificial Intelligence (2019, background): “This requirement is still meaningful even if intelligence is (as it should be) considered as a matter of degree (Hernández-Orallo, 2017).”
Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research (2019, background): “Intelligence may be defined as the ability to adapt to a diverse set of complex environments (Hernández-Orallo, 2017; Legg and Hutter, 2007).”
The Measure of Intelligence (2019, unknown): “A decade later, in 2017, Hernández-Orallo released an extensive survey of evaluation methods [36] as well as a comprehensive book on AI evaluation [37].”
α-Rank: Multi-Agent Evaluation by Evolution (2019, background): “Researchers have also carried out studies of evaluation metrics in the fields of computer science, machine learning, and artificial intelligence [90–92].”
Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks (2020, methodology): “When aiming at evaluating the progress in a speciﬁc (AI) discipline, we need to focus on objective evaluation tools to measure the elements and objects of study, assess the prototypes and artefacts that are being built and examine the discipline as a whole (Hernández-Orallo, 2017b).”
A Statistical Turing Test for Generative Models (2023, background): “, comparing a system’s accuracy to human-level accuracy) terms – countless times throughout our history [18].”