The Minds of Many: Opponent Modeling in a Stochastic Game

By Friedrich Burkhard von der Osten, Michael Kirley, Tim Miller. 2017

37 citations recorded by the source corpus.

The Theory of Mind provides a framework for an agent to predict the actions of adversaries by building an abstract model of their strategies using recursive nested beliefs. In this paper, we extend a recently introduced technique for opponent modelling based on Theory of Mind reasoning. Our extended multi-agent Theory of Mind model explicitly considers multiple opponents simultaneously. We introduce a stereotyping mechanism, which segments the agent population into sub-groups of agents with similar behaviour. Sub-group profiles guide decision making. We evaluate our model using a multi-player stochastic game, which presents agents with the challenge of unknown adversaries in a partially-observable environment. Simulation results demonstrate that the model performs well under uncertainty and that stereotyping allows larger groups of agents to be modelled robustly. The findings show that Theory of Mind modelling is useful in many artificial intelligence applications.

Subfields: Safety Research, Sociology and Political Science, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety Research

1. Pablo Hernandez-Leal
2. Tim Miller

In Sociology and Political Science

1. Pablo Hernandez-Leal
2. Tim Miller
3. Michael Kaisers

In Statistical and Nonlinear Physics

1. Pablo Hernandez-Leal
2. Michael Kaisers
3. Enrique Munoz de Cote

Citations in the corpus

• Explanation in Artificial Intelligence: Insights from the Social Sciences (2017, background): “While the idea of Theory of Mind in AI is not new; see for example [177, 37]; it’s application to explanation has not been adequately explored.”
• Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learning (2019, unknown): “Despite some initial trails (Gmytrasiewicz & Doshi, 2005; Von Der Osten et al., 2017), there has been little work that tries to adopt this idea into the multi-agent deep reinforcement learning (DRL) setting.”
• A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity (2017, background): “Recently, an extension to more than one opponent, multiagent ToM (MToM), was proposed by Van der Osten et al. (2017).”