Reconciling deep learning with symbolic artificial intelligence: representing objects and relations

By Marta Garnelo, Murray Shanahan. 2019

237 citations recorded by the source corpus.

In the history of the quest for human-level artificial intelligence, a number of rival paradigms have vied for supremacy. Symbolic artificial intelligence was dominant for much of the 20th century, but currently a connectionist paradigm is in the ascendant, namely machine learning with deep neural networks. However, both paradigms have strengths and weaknesses, and a significant challenge for the field today is to effect a reconciliation. A central tenet of the symbolic paradigm is that intelligence results from the manipulation of abstract compositional representations whose elements stand for objects and relations. If this is correct, then a key objective for deep learning is to develop architectures capable of discovering objects and relations in raw data, and learning how to represent them in ways that are useful for downstream processing. This short review highlights recent progress in this direction.

Subfields: Artificial Intelligence, Cultural Studies, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cultural Studies

In Representation & Self-Supervised Learning

Citations in the corpus

Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning (2022, background): “…of the most valued strengths of classical, symbolic AI over contemporary deep learning methods (Bengio et al., 2021; Marcus, 2020; Marcus and Davis, 2019), prompting the recent increase in the use of neurosymbolic approaches to bridge this gap (Garcez and Lamb, 2020; Garnelo and Shanahan, 2019).”
Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, positive): “Combining connectionist and symbolic paradigms seems a favourable way to address this challenge [169,299,312,352,353] .”
On the Binding Problem in Artificial Neural Networks (2020, background): “Smolensky, 1990; Bader and Hitzler, 2005), especially in recent years (e.g. Lake et al., 2017; Battaglia et al., 2018; Hamrick, 2019; Garnelo and Shanahan, 2019).”
Learning First-Order Symbolic Representations for Planning from the Structure of the State Space (2019, background): “Recent work in deep symbolic relational reinforcement learning [16] attempts to account for objects and relations through the use of attention mechanisms and loss functions but the semantic and conceptual gap between the low-level techniques and the high-level representations that are required…”
Neurosymbolic AI: the 3rd wave (2020, unknown): “Key to the appreciation of the above statement by Marcus, also advocated in [46] and [28], is an understanding of the representational value of the symbolic manipulation of variables in logic and the compositionality of language.”
Symbolic Behaviour in Artificial Intelligence (2021, background): “The ongoing debate about symbols in AI mostly revolves around the properties of certain ways of computing [9, 10, 11, 12, 13].”
Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks (2020, methodology): “In contrast, the generalizing algorithm for solving reasoning problems requires proper variable manipulation (Garnelo & Shanahan, 2019).”
Hierarchical Relational Inference (2020, background): “This is known to affect their ability to accurately model physical interactions and extrapolation (van Steenkiste et al. 2018; Garnelo and Shanahan 2019), despite their remarkable capacity for modeling more complex visually scenes.”
Explainability in Deep Reinforcement Learning (2020, background): “Hierarchical learning [23] [110] [15] [103] Simultaneous leaning [53] [71] [107] [70] Representation learning [111] [24] [84] [85] [38] [24] [80]”
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next (2022, background): “The model representations could be built up directly from a small amount of data with some priors (Garnelo and Shanahan, 2019).”
A historical perspective of explainable Artificial Intelligence (2020, background): “In addition to this, learning with logical constraints generally has shown to be beneficial for improving the data efficiency (Garnelo & Shanahan, 2019).”
Differentiable Reasoning on Large Knowledge Bases and Natural Language (2019, background): “While symbolic models can generalise well from a small number of examples, they are brittle and prone to failure when the observations are noisy or ambiguous, or when the properties of the domain are unknown or hard to formalise, all of which being the case for natural language (Raedt et al. 2008; G”