Machine Theory of Mind

By Neil C. Rabinowitz, Frank Perbet, H. Francis Song, Chiyuan Zhang, S. M. Ali Eslami, Matthew Botvinick. 2018

613 citations recorded by the source corpus.

Theory of mind (ToM; Premack & Woodruff, 1978) broadly refers to humans' ability to represent the mental states of others, including their desires, beliefs, and intentions. We propose to train a machine to build such models too. We design a Theory of Mind neural network -- a ToMnet -- which uses meta-learning to build models of the agents it encounters, from observations of their behaviour alone. Through this process, it acquires a strong prior model for agents' behaviour, as well as the ability to bootstrap to richer predictions about agents' characteristics and mental states using only a small number of behavioural observations. We apply the ToMnet to agents behaving in simple gridworld environments, showing that it learns to model random, algorithmic, and deep reinforcement learning agents from varied populations, and that it passes classic ToM tasks such as the "Sally-Anne" test (Wimmer & Perner, 1983; Baron-Cohen et al., 1985) of recognising that others can hold false beliefs about the world. We argue that this system -- which autonomously learns how to model other agents in its world -- is an important step forward for developing multi-agent AI systems, for building intermediating technology for machine-human interaction, and for advancing the progress on interpretable AI.

Subfields: Artificial Intelligence, Sociology and Political Science, Large Language Models, Prompting & Reasoning, Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Sociology and Political Science

In Large Language Models

In Prompting & Reasoning

In Cognitive Science & Psychology

Citations in the corpus

The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence (2020, background): “§ The good news is that although nativism has often been frowned on in the deep learning community, historically, there are some growing signs of a greater openness to “priors” (e.g., Burgess et al., 2019; Rabinowitz et al., 2018; Maier et al., 2017).”
Evaluating Theory of Mind in Question Answering (2018, background): “Rabinowitz et al. (2018) formulate the capacity to reason about others’ beliefs as a meta-learning problem.”
Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, background): “In contrast to the diverse set of scenarios included in SMAC, the Hanabi Challenge focusses on a single task involving between 2-5 agents designed to test agents’ ability to reason about the actions of the other agents (an ability in humans referred to as theory of mind (Rabinowitz et al., 2018)).”
Continuous Coordination As a Realistic Scenario for Lifelong Learning (2021, background): “Thus, Hanabi is a challenging game that requires the agent to possess the theory of mind (Premack & Woodruff, 1978; Rabinowitz et al., 2018) in order to cooperate effectively.”
The Hanabi Challenge: A New Frontier for AI Research (2019, background): “signiﬁcant ﬁnancial support for this work that could have inﬂuenced its outcome.”
QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2018, methodology): “agents designed to test agents’ ability to reason about the 1. Code is available at https://github.com/oxwhirl/smac. 3 actions of the other agents (an ability in humans referred to as theory of mind (Rabinowitz et al., 2018)). To further facilitate research in this eld, we also open-source PyMARL, a”
Negative Update Intervals in Deep Multi-Agent Reinforcement Learning (2018, methodology): “Ls develop this adaptive ability. 2) For simplicity, the actions u ∈U returned by the oracle ϑfor the AFG are predefined. Going foward, recent work on the topic of theory of mind by Rabinowitz et al. [22] could pave the way for a learnedoracleϑ.Theauthorsbuildadata-efficientmeta-learnerthat learns m”
Relational Forward Models for Multi-Agent Learning (2018, neutral): “…analysis tools that allow researchers to understand what drives agents’ behavior, what are the most common failure modes and provide insights into the inner workings of learning systems (Zeiler & Fergus, 2013; Yosinski et al., 2015; Olah et al., 2017; Rabinowitz et al., 2018; Morcos et al., 2018).”
Emergent Communication through Negotiation (2018, background): “Intuitively, the induced agent embedding, that takes the form of machine theory of mind (Rabinowitz et al., 2018), should capture any information about the opponent’s behaviour (e.g., negotiating strategy) that, when taken into account, can help an agent maximize their reward.”
Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning (2019, methodology): “Architectures such as graph neural networks (Tacchetti et al. , 2019), feed-forward neural networks (Grover et al. , 2018), and recurrent neural networks (Rabinowitz et al. , 2018) can be used to produce the representations.”
A systematic review of algorithm aversion in augmented decision making (2019, methodology): “Although research around theory of mind has predominantly been reserved to developmental psychology domains, its applicability to artificial intelligence and robotics has been recognized in recent work (e.g., Rabinowitz et al., 2018; Winfield, 2018).”
Theory of Mind and Preference Learning at the Interface of Cognitive Science, Neuroscience, and AI: A Review (2022, background): “Rabinowitz et al. (2018) shows a proof of concept of machine prediction of other agents’ false beliefs, inspired by the cognitive Sally-Anne test in humans, by formulating ToM as a meta-learning problem.”
Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration (2020, background): “humans’ goals by observing their actions (Baker et al., 2017; Ullman et al., 2009; Rabinowitz et al., 2018; Shum et al., 2019).”
Multi-agent deep reinforcement learning: a survey (2021, methodology): “Meta-learning can also be utilized to construct agent models (Rabinowitz et al. 2018).”
Learning to Communicate Implicitly by Actions (2018, background): “In this work, we develop agents that benefit from considering others’ perspectives and thereby explore the further development of machine theory of mind (Rabinowitz et al. 2018).”
Interactive Agent Modeling by Learning to Probe (2018, methodology): “Rabinowitz et al. (2018) adopts a simpler mind representation (i.e., a latent vector) learned by a neural net (ToMnet).”
Revisiting the Evaluation of Theory of Mind through Question Answering (2019, background): “We believe this is a promising approach for advancing research on theory of mind, as it decouples its evaluation from problems such as multi-agent systems, game theory, and meta-learning, which are all components of other evaluation methods (Rabinowitz et al., 2018; Bard et al., 2019).”