Neurosymbolic AI: the 3rd wave

By A.S. d'Avila Garcez, L. C. Lamb. 2020

562 citations recorded by the source corpus.

Current advances in Artificial Intelligence (AI) and Machine Learning have achieved unprecedented impact across research communities and industry. Nevertheless, concerns around trust, safety, interpretability and accountability of AI were raised by influential thinkers. Many identified the need for well-founded knowledge representation and reasoning to be integrated with deep learning and for sound explainability. Neurosymbolic computing has been an active area of research for many years seeking to bring together robust learning in neural networks with reasoning and explainability by offering symbolic representations for neural models. In this paper, we relate recent and early research in neurosymbolic AI with the objective of identifying the most important ingredients of neurosymbolic AI systems. We focus on research that integrates in a principled way neural network-based learning with symbolic knowledge representation and logical reasoning. Finally, this review identifies promising directions and challenges for the next decade of AI research from the perspective of neurosymbolic computing, commonsense reasoning and causal explanation.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Safety, Robustness & Interpretability

Citations in the corpus

Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning (2022, background): “There are many ﬂavours of neurosymbolic models (Garcez and Lamb, 2020).”
PAL: Program-aided Language Models (2022, background): “Until as recently as three years ago, reasoning was considered to be one of the most significant challenges that large language models (LLMs) had not yet overcome (Marcus, 2018; 2020; Garcez & Lamb, 2020).”
Tell me why! Explanations support learning relational and causal structure (2021, background): “Our approach does not force the agent to use explanations directly, and therefore might be less harmful in such cases than stronger constraints like requiring symbolic representations (e.g., Garcez & Lamb, 2020).”
On the link between conscious function and general intelligence in humans and machines (2022, background): “Despite the debate over the importance of neurosymbolic approaches (Marcus, 2020; Garcez & Lamb, 2020), for the time being it seems that simply scaling architectures and data results in models that are able to reason over progressively higher-level concepts (Brown et al., 2020; Radford et al.,…”
Large Language Models (2023, background): “Given this point of view one can seek a synthesis or “neurosymbolic” approach, advocated in many works [7, 47, 72].”
D’ya Like DAGs? A Survey on Structure Learning and Causal Discovery (2021, background): “It is also a burgeoning topic in machine learning and artificial intelligence [6, 16, 65, 75, 144, 214, 255, 265], where it has been argued that a consideration for causality is crucial for reasoning about the world.”
Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning (2021, methodology): “It has been argued that integrating System 2 processing could similarly improve AI systems (Goyal & Bengio, 2020; Garcez & Lamb, 2020), and here we explore this idea as applied to neural sequence models.”
Informed Machine Learning – A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems (2019, background): “connectionist and symbolic systems into hybrid systems is a longtime idea [140], [141], it is currently getting more attention [142], [143].”
Neurosymbolic Program Synthesis (unknown): “Neurosymbolic machine learning [1,2,3] is an increasingly prominent response to these concerns.”
Tell me why! - Explanations support learning of relational and causal structure (2021, background): “Our approach does not force the agent to use explanations directly, and therefore might be less harmful in such cases than stronger constraints like requiring symbolic representations (Garcez and Lamb, 2020).”
Symbolic Behaviour in Artificial Intelligence (2021, background): “The ongoing debate about symbols in AI mostly revolves around the properties of certain ways of computing [9, 10, 11, 12, 13].”