The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence

By Gary F. Marcus. 2020

495 citations recorded by the source corpus.

Recent research in artificial intelligence and machine learning has largely emphasized general-purpose learning and ever-larger training sets and more and more compute. In contrast, I propose a hybrid, knowledge-driven, reasoning-based approach, centered around cognitive models, that could provide the substrate for a richer, more robust AI than is currently possible.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020, background): “While this development is exciting, such models do have downsides: They cannot easily expand or revise their memory, can’t straightforwardly provide insight into their predictions, and may produce “hallucinations” [34].”
Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning (2022, background): “It is also one of the most valued strengths of classical, symbolic AI over contemporary deep learning methods (Marcus & Davis, 2019; Marcus, 2020; Bengio et al., 2021), prompting the recent increase in the use of neurosymbolic approaches to bridge this gap (Garnelo & Shanahan, 2019; Garcez & Lamb, 2”
PAL: Program-aided Language Models (2022, background): “Until as recently as three years ago, reasoning was considered to be one of the most significant challenges that large language models (LLMs) had not yet overcome (Marcus, 2018; 2020; Garcez & Lamb, 2020).”
Out of One, Many: Using Language Models to Simulate Human Samples (2022, background): “Many of the known shortcomings and inaccuracies of large language models still apply (Bender et al. 2021; Marcus 2020).”
Neurosymbolic AI: the 3rd wave (2020, unknown): “Kahneman also stated that S2 (as the sequential deliberative system) is most probably performing symbol manipulation as argued by Marcus (2020).”
The AI gambit: leveraging artificial intelligence to combat climate change—opportunities, challenges, and recommendations (2021, methodology): “Neverthe-less, we note that research exploring new neural network architectures or new hardware – software – algorithm combinations has largely been side-lined in favour of compute-intensive AI research (Hooker 2020; Marcus 2020; Ahmed and Wahed 2020).”
Tell me why! Explanations support learning relational and causal structure (2021, background): “It is often argued that machine learning models—and deep learning models in particular—lack the human proficiencies for forming abstractions and inferring relational or causal structure (e.g. Fodor & Pylyshyn, 1988; Lake et al., 2017; Pearl, 2018; Marcus, 2020; Ichien et al., 2021; Holyoak & Lu, 202”
Tell me why! - Explanations support learning of relational and causal structure (2021, background): “…and analogies, and generalization—are often cited as areas where deep learning has not yet achieved human-level performance (e.g. Fodor and Pylyshyn, 1988; Lake et al., 2017; Pearl, 2018; Marcus, 2020; Ichien et al., 2021; Holyoak and Lu, 2021; Puebla and Bowers, 2021; Geirhos et al., 2020).”
Symbolic Behaviour in Artificial Intelligence (2021, background): “The ongoing debate about symbols in AI mostly revolves around the properties of certain ways of computing [9, 10, 11, 12, 13].”
Learning Decision Trees Recurrently Through Communication (2021, background): “Beyond explainable ML, a sparse representation is considered to be essential for moving towards hybrid deep learningsymbolic models [11, 44] and for obtaining representations that are closer to conscious reasoning [2].”
On the link between conscious function and general intelligence in humans and machines (2022, background): “symbolic reasoning capabilities into neural networks (Marcus, 2020), we will discuss what “standard” neural networks are capable of already.”
On scientific understanding with artificial intelligence (2022, background): “[8] Gary Marcus, “The next decade in ai: four steps towards robust artificial intelligence,” arXiv:2002.”
Towards Reasoning in Large Language Models: A Survey (2022, background): “frequently considered missed in current artificial intelligence systems (Marcus, 2020; Russin et al., 2020; Mitchell, 2021; Bommasani et al., 2021).”
LAMBADA: Backward Chaining for Automated Reasoning in Natural Language (2022, background): “One solution to the aforementioned problems is to integrate the strength and reliability of classical AI models in logical reasoning with LMs (Garcez and Lamb, 2020; Marcus, 2020).”
How Much Do Language Models Copy From Their Training Data? Evaluating Linguistic Novelty in Text Generation Using RAVEN (2021, unknown): “Much discussion about evaluating LMs focuses on whether they understand language (Bender and Koller, 2020; Marcus, 2020), whereas we assess the novelty of surface text.”
Discovering Symbolic Models from Deep Learning with Inductive Biases (2020, background): “Such an idea is discussed in detail in [45].”