Emergent Communication through Negotiation

By Kris Cao, Angeliki Lazaridou, Marc Lanctot, Joel Z. Leibo, Karl Tuyls, Stephen Clark. 2018

173 citations recorded by the source corpus.

Multi-agent reinforcement learning offers a way to study how communication could emerge in communities of agents needing to solve specific problems. In this paper, we study the emergence of communication in the negotiation environment, a semi-cooperative model of agent interaction. We introduce two communication protocols -- one grounded in the semantics of the game, and one which is \textit{a priori} ungrounded and is a form of cheap talk. We show that self-interested agents can use the pre-grounded communication channel to negotiate fairly, but are unable to effectively use the ungrounded channel. However, prosocial agents do learn to use cheap talk to find an optimal negotiating strategy, suggesting that cooperation is necessary for language to emerge. We also study communication behaviour in a setting where one agent interacts with agents in a community with different levels of prosociality and show how agent identifiability can aid negotiation.

Subfields: Management Science and Operations Research, Sociology and Political Science, Statistical and Nonlinear Physics, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

In Sociology and Political Science

In Statistical and Nonlinear Physics

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering (2019, background): “This can also be seen as an instance of two agents cooperating via communication to solve a task (Lazaridou et al., 2017; Lee et al., 2018; Cao et al., 2018).”
OpenSpiel: A Framework for Reinforcement Learning in Games (2019, neutral): “Game Reference(s) Backgammon Wikipedia Breakthrough Wikipedia Bridge bidding Wikipedia Catch [51] and [59, Appendix A] Coin Game [64] Connect Four Wikipedia Cooperative Box-Pushing [70] Chess Wikipedia First-price Sealed-bid Auction Wikipedia Go Wikipedia Goofspiel Wikipedia Hanabi (via HLE) Wikiped”
The Hanabi Challenge: A New Frontier for AI Research (2019, background): “hms have been developed to learn to solve riddles and referential games [6, 73], gridworld games requiring coordination [74], object identication via question-and-answer dialog [75], and negotiation [7, 8]. The main dierence in Hanabi is that there is no \cheap-talk" communication channel: any ”
XOC: Explainable Observer-Classifier for Explainable Binary Decisions (2019, background): “Learning to communicate in a multi-agent setting has gained interest with the emergence of deep reinforcement learning [9, 10, 22, 4, 15, 7, 6].”
Learning Decision Trees Recurrently Through Communication (2021, background): “[15] and [5] train multiple agents to maximize a shared utility by establishing their own language.”
Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning (2018, background): “…emergent communication protocols also remains a challenging problem; recent empirical results underscore the difficulty of learning meaningful emergent communication protocols, even when relying on centralized training (e.g., Lazaridou et al. (2018); Cao et al. (2018); Foerster et al. (2016)).”
Multi-agent deep reinforcement learning: a survey (2021, background): “Similar studies concerned with negotiations are covered in Cao et al. (2018) and He et al. (2018).”
CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems (2021, unknown): “Most efforts focused on agent-agent interactions (Williams et al., 2012; Lin et al., 2014; Cao et al., 2018), although there is an increasing interest in human-agent negotiations (Mell and Gratch, 2017) as well.”
A survey of multi-agent deep reinforcement learning with communication (2022, unknown): “The emergent language research area [18, 44, 45, 46, 47] aims at learning a language grounded on symbols in communities of interacting/communicating agents.”
Decoupling Strategy and Generation in Negotiation Dialogues (2018, background): “Most past work in NLP on negotiation focuses on strategy (dialogue management) with either no natural language (Cuayáhuitl et al., 2015; Cao et al., 2018) or canned responses (Keizer et al.”
Actor-Critic Policy Optimization in Partially Observable Multiagent Environments (2018, background): “There has also been progress for mixed cooperative/competitive environments: using a centralized critic [57], learning to negotiate [18], anticipating/learning opponent responses in social dilemmas [23, 53], and control in realistic physical environments [3, 7].”
Few-shot Language Coordination by Modeling Theory of Mind (2021, methodology): “Following previous work on communicative agents (Lazaridou et al., 2016; Cao et al., 2018; Lowe et al., 2019a), we use goal-oriented language games as the test bed for few-shot language coordination.”