Monte Carlo Planning Method Estimates Planning Horizons during Interactive Social Exchange

By Andreas Hula, P. R. Montague, Peter S. Dayan. 2015

102 citations recorded by the source corpus.

Reciprocating interactions represent a central feature of all human exchanges. They have been the target of various recent experiments, with healthy participants and psychiatric populations engaging as dyads in multi-round exchanges such as a repeated trust task. Behaviour in such exchanges involves complexities related to each agent’s preference for equity with their partner, beliefs about the partner’s appetite for equity, beliefs about the partner’s model of their partner, and so on. Agents may also plan different numbers of steps into the future. Providing a computationally precise account of the behaviour is an essential step towards understanding what underlies choices. A natural framework for this is that of an interactive partially observable Markov decision process (IPOMDP). However, the various complexities make IPOMDPs inordinately computationally challenging. Here, we show how to approximate the solution for the multi-round trust task using a variant of the Monte-Carlo tree search algorithm. We demonstrate that the algorithm is efficient and effective, and therefore can be used to invert observations of behavioural choices. We use generated behaviour to elucidate the richness and sophistication of interactive inference.

Subfields: General Decision Sciences, Safety Research, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In General Decision Sciences

1. Brenna D. Argall
2. Stuart Armstrong
3. Sören Mindermann
4. Siddarth Jain
5. Matthew Botvinick
6. Sergey Levine

In Safety Research

1. Brenna D. Argall
2. Stuart Armstrong
3. Siddarth Jain
4. Matthew Botvinick

In Sociology and Political Science

1. Brenna D. Argall
2. Stuart Armstrong
3. Siddarth Jain
4. Matthew Botvinick

Citations in the corpus

• Machine Theory of Mind (2018, background): “Learning rich models of others will improve decision-making in complex multi-agent tasks, especially where model-based planning and imagination are required (Hassabis et al., 2013; Hula et al., 2015; Oliehoek & Amato, 2016).”
• Occam's razor is insufficient to infer the preferences of irrational agents (2017, background): “Noisy rationality explains human be-5 havior on various data sets better [Hula et al., 2015].”
• Probabilistic Human Intent Recognition for Shared Autonomy in Assistive Robotics (2019, methodology): “We model these actions as observations using Boltzmann rationality, which has been shown to explain human behavior on various datasets [15].”