Empirical evaluation of ad hoc teamwork in the pursuit domain

By Samuel Barrett, Peter Stone, Sarit Kraus. 2011

159 citations recorded by the source corpus.

The concept of creating autonomous agents capable of exhibiting ad hoc teamwork was recently introduced as a challenge to the AI, and specifically to the multiagent systems community. An agent capable of ad hoc teamwork is one that can effectively cooperate with multiple potential teammates on a set of collaborative tasks. Previous research has investigated theoretically optimal ad hoc teamwork strategies in restrictive settings. This paper presents the first empirical study of ad hoc teamwork in a more open, complex teamwork domain. Specifically, we evaluate a range of effective algorithms for on-line behavior generation on the part of a single ad hoc team agent that must collaborate with a range of possible teammates in the pursuit domain.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

"Other-Play" for Zero-Shot Coordination (2020, background): “A closely related problem to zero-shot coordination is adhoc teamwork (Stone et al., 2010; Barrett et al., 2011).”
AI-QMIX: Attention and Imagination for Dynamic Multi-Agent Reinforcement Learning (2020, background): “While existing lines of research have explored curriculum learning to scale from small to larger scenarios [1, 21, 36] or collaboration with agents not seen during training [31, 4, 14], to our knowledge, no other works have explored the setting of simultaneous learning across diverse scenarios.”
Comparative evaluation of MAL algorithms in a diverse set of ad hoc team problems (2012, background): “Second, virtually all MAL algorithms which were specifically designed for ad hoc team problems (e.g. [ 2,7,30,31,36]) were primarily evaluated in teams of agents with ﬁxed behaviours.”
Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning (2020, methodology): “WOLFPACK To create a diverse set of teammates for open ad hoc teamwork, we used the following mixture between heuristics proposed by Barrett et al. (2011) for the predator prey domain along with RL-based models to control teammates : • Random agent (H1): The random agent chooses its action at any…”
Evaluating and modelling Hanabi-playing agents (2017, methodology): “These results are consistent with the ﬁndings of [7].”
Sidekick agents for sequential planning problems (2013, methodology): “A similar strategy using approximations from state-space planning has also shown success in cooperative planning in teams with unknown team mates [4].”
Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning (2020, background): “ly complex environments. This setting requires policies to perform well across various scenarios, potentially including those that the agents have never seen. Research on the topic of ad hoc teamwork [31, 4] is closely related to our setting, though with some key differences. Similar to our work, ad”
Autonomous agents modelling other agents: A comprehensive survey and open problems (2017, methodology): “Moreover, since types are essentially blackbox mappings, they can encapsulate policy reconstruction methods to learn new types during the interaction [10,33].”
Combining reward shaping and hierarchies for scaling to large multiagent systems (2016, unknown): “…have been presented to address these issues including organizational structures, shaped rewards, learning automata, and coordination graphs (Horling & Lesser, 2005; Tambe et al ., 2005; Xu et al ., 2005; Kok & Vlassis, 2006; Vrancx et al ., 2008; Barrett et al ., 2011; Howley & Duggan, 2011).”
Evolving Agents for the Hanabi 2018 CIG Competition (2018, background): “We are particularly interested in initiatives such as [20], which attempts to build a model of unknown partners during cooperative gameplay by interpolating between known models.”
Reasoning about Hypothetical Agent Behaviours and their Parameters (2017, methodology): “Some methods assume discrete (usually finite) type spaces [3, 9, 14] while others assume continuous type spaces [15,36].”