Ad Hoc Autonomous Agent Teams: Collaboration without Pre-Coordination

By Peter Stone, Gal A. Kaminka, Sarit Kraus, Jeffrey S. Rosenschein. 2010

497 citations recorded by the source corpus.

As autonomous agents proliferate in the real world, both in software and robotic settings, they will increasingly need to band together for cooperative activities with previously unfamiliar teammates. In such ad hoc team settings, team strategies cannot be developed a priori. Rather, an agent must be prepared to cooperate with many types of teammates: it must collaborate without pre-coordination. This paper challenges the AI community to develop theory and to implement prototypes of ad hoc team agents. It defines the concept of ad hoc team agents, specifies an evaluation paradigm, and provides examples of possible theoretical and empirical approaches to challenge. The goal is to encourage progress towards this ambitious, newly realistic, and increasingly important research goal.

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

Continuous Coordination As a Realistic Scenario for Lifelong Learning (2021, background): “Stone et al. (2010) ﬁrst introduced ad-hoc team-work challenge as a multi-agent task where autonomous agents need to collaborate with previously unknown teammates on tasks in which they are all individually capable of contributing as team members as well as characteristics of a good ad-hoc player.”
The Hanabi Challenge: A New Frontier for AI Research (2019, background): “Notably, humans collaborate successfully without relying on prior coordination or knowing the policy of their teammates, commonly referred to as the ad-hoc team setting [16, 17].”
Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery (2019, background): “We investigated the test performance of agents in ad-hoc cooperation, by giving them teammate(s) with whom they never previously trained [34].”
Human-level performance in 3D multiplayer games with population-based reinforcement learning (2018, methodology): “We aimed to devise an algorithm and training procedure that enables agents to acquire policies that are robust to the variability of maps, number of players, and choice of teammates and opponents, a challenge that generalizes that of ad hoc teamwork (24).”
A survey and critique of multiagent deep reinforcement learning (2018, background): “ve their own cards but other players can reveal information about them. This makes an interesting challenge for learning algorithms in particular in the context of self-play learning and ad-hoc teams [306, 307, 308]. The Hanabi Learning Environment [309] was recently released24 and it is accompanied”
"Other-Play" for Zero-Shot Coordination (2020, background): “A closely related problem to zero-shot coordination is adhoc teamwork (Stone et al., 2010; Barrett et al., 2011).”
Local Information Agent Modelling in Partially-Observable Environments (2021, background): “Agents may only have a limited view of their surroundings, communication with other agents may be infeasible or unreliable [Stone et al., 2010], and knowledge of the perception system of other agents may be unavailable [Gmytrasiewicz and Doshi, 2005].”
Comparative evaluation of MAL algorithms in a diverse set of ad hoc team problems (2012, methodology): “In a version of this setting, called the ad hoc team problem [29], one seeks to design an autonomous agent which is able to collaborate eﬃciently with a previously unknown group of agents, in the absence of any prior coordination between the agent and its counterparts chosen in an ad hoc way.”
Human-Timescale Adaptation in an Open-Ended Task Space (2023, background): “When faced with a new problem, agents must adapt on-the-ﬂy to agree on a single equilibrium of mutual beneﬁt (Stone et al., 2010; Hu et al., 2020; Christianos et al., 2022).”
Empirical evaluation of ad hoc teamwork in the pursuit domain (2011, background): “[15], and there have been a few theoretical analyses of specific applications of ad hoc teams [16, 17].”
Autonomous agents modelling other agents: A comprehensive survey and open problems (2017, background): “In AI research, type-based methods6 found popularity in problems of multiagent interaction without prior coordination (Albrecht et al., 2017; Stone et al., 2010), in which the controlled agent interacts with other agents whose behaviours are initially unknown.”
Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning (2020, background): “Thus, research in ad hoc teamwork (Stone et al., 2010) aims to design a single autonomous agent, which we refer to as the learner, that can interact effectively with other agents without pre-coordination such as joint training.”
AI-QMIX: Attention and Imagination for Dynamic Multi-Agent Reinforcement Learning (2020, background): “While existing lines of research have explored curriculum learning to scale from small to larger scenarios [1, 21, 36] or collaboration with agents not seen during training [31, 4, 14], to our knowledge, no other works have explored the setting of simultaneous learning across diverse scenarios.”
Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning (2020, background): “ly complex environments. This setting requires policies to perform well across various scenarios, potentially including those that the agents have never seen. Research on the topic of ad hoc teamwork [31, 4] is closely related to our setting, though with some key differences. Similar to our work, ad”
Diverse Agents for Ad-Hoc Cooperation in Hanabi (2019, background): “While the general problem of ad-hoc cooperation has been addressed at least since [6], for the rest of this paper we will focus on ad-hoc cooperation applied to Hanabi.”
Three years of the RoboCup standard platform league drop-in player competition (2016, unknown): “imagined ‘staging field tests of ad hoc team agents at the annual RoboCup competitions’ in their 2010 AAAI challenge paper that introduced ad hoc teamwork [21].”
Off-Belief Learning (2021, background): “Finally we ﬁnd it also plays well in ad-hoc team play (Stone et al., 2010) when paired with an agent trained by imitation learning from human data and two distinct agents trained with reinforcement learning, all of which are unseen during training time.”
A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity (2017, background): “Stone et al. (2010) presented the challenge of ad-hoc teamworks, this is, to create an autonomous agent that is able to efficiently and robustly collaborate with previously unknown teammates on tasks to which they are all individually capable of contributing as team members.”