Reasoning about Hypothetical Agent Behaviours and their Parameters

By Stefano V. Albrecht, Peter Stone. 2017

76 citations recorded by the source corpus.

Agents can achieve effective interaction with previously unknown other agents by maintaining beliefs over a set of hypothetical behaviours, or types, that these agents may have. A current limitation in this method is that it does not recognise parameters within type specifications, because types are viewed as blackbox mappings from interaction histories to probability distributions over actions. In this work, we propose a general method which allows an agent to reason about both the relative likelihood of types and the values of any bounded continuous parameters within types. The method maintains individual parameter estimates for each type and selectively updates the estimates for some types after each observation. We propose different methods for the selection of types and the estimation of parameter values. The proposed methods are evaluated in detailed experiments, showing that updating the parameter estimates of a single type after each observation can be sufficient to achieve good performance.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks (2020, background): “In Level-Based Foraging (LBF) [Albrecht and Ramamoorthy, 2013, Albrecht and Stone, 2017] agents must collect food items which are scattered randomly in a grid-world.”
Multi-Agent Common Knowledge Reinforcement Learning (2018, background): “gmon, and Stone (2013) and Barrett, Stone, and Kraus (2011) examine ad hoc teamwork: how agents can cooperate with previously unseen teammates when there are a variable number of non-learning agents. Albrecht and Stone (2017) address ad hoc teamwork by maintaining a belief over a set of parameterise”
Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning (2020, methodology): “…different types of heuristics used in level-based foraging are provided below: • Heuristic H1: This type of agent follows heuristic θF2j proposed by Albrecht & Stone (2017) where agents under this heuristic follow the agent with the highest level if it observes another agent with a higher level…”
Local Information Agent Modelling in Partially-Observable Environments (2021, methodology): “We evaluate LIAM in three different multi-agent environments: double speaker-listener [Mordatch and Abbeel, 2017, Lowe et al., 2017], level-based foraging (LBF) [Albrecht and Stone, 2017, Papoudakis et al., 2020], and a modiﬁed version of predator-prey proposed by [Böhmer et al., 2020].”
Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning (2020, background): “2c: LBF [1, 3] is a mixed cooperative-competitive game which focuses on the coordination of the agents involved.”
Autonomous agents modelling other agents: A comprehensive survey and open problems (2017, methodology): “Essentially the same method can be used to model agents which condition their choices on abstract feature vectors derived from the history (Chakraborty and Stone, 2013).”