MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks

By Cristian-Paul Bara, Sky CH-Wang, Joyce Yue Chai. 2021

83 citations recorded by the source corpus.

An ideal integration of autonomous agents in a human world implies that they are able to collaborate on human terms. In particular, theory of mind plays an important role in maintaining common ground during human collaboration and communication. To enable theory of mind modeling in situated interactions, we introduce a fine-grained dataset of collaborative tasks performed by pairs of human subjects in the 3D virtual blocks world of Minecraft. It provides information that captures partners’ beliefs of the world and of each other as an interaction unfolds, bringing abundant opportunities to study human collaborative behaviors in situated language communication. As a first step towards our goal of developing embodied AI agents able to infer belief states of collaborative partners in situ, we build and present results on computational models for several theory of mind tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Social Psychology, Dialogue & Conversational AI, Cognitive Science & Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Social Psychology

In Cognitive Science & Psychology

Citations in the corpus

Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs (2022, background): “require reasoning about mental states, could help with neural Theory of Mind (Bara et al., 2021).”