Decoupling Strategy and Generation in Negotiation Dialogues

By He He, Derek Chen, Anusha Balakrishnan, Percy Liang. 2018

249 citations recorded by the source corpus.

We consider negotiation settings in which two agents use natural language to bargain on goods. Agents need to decide on both high-level strategy (e.g., proposing $50) and the execution of that strategy (e.g., generating “The bike is brand new. Selling for just $50!”). Recent work on negotiation trains neural models, but their end-to-end nature makes it hard to control their strategy, and reinforcement learning tends to lead to degenerate solutions. In this paper, we propose a modular approach based on coarse dialogue acts (e.g., propose(price=50)) that decouples strategy and generation. We show that we can flexibly set the strategy using supervised learning, reinforcement learning, or domain-specific knowledge without degeneracy, while our retrieval-based generation can maintain context-awareness and produce diverse utterances. We test our approach on the recently proposed DEALORNODEAL game, and we also collect a richer dataset based on real items on Craigslist. Human evaluation shows that our systems achieve higher task success rate and more human-like negotiation behavior than previous approaches.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Experience Grounds Language (2020, background): “In order to learn the effects language has on the world, an agent must participate in linguistic activity, such as negotiation (Yang et al., 2019a; He et al., 2018; Lewis et al., 2017), collaboration (Chai et al.”
Recommendation as a Communication Game: Self-Supervised Bot-Play for Goal-oriented Dialogue (2019, background): “Other dialogue settings with shared objectives have been explored, for example a collaborative graph prediction task (He et al., 2017a), and semi-cooperative negotiation tasks (Lewis et al., 2017; Yarats and Lewis, 2018; He et al., 2018).”
MOSS: End-to-End Dialog System Framework with Modular Supervision (2019, background): “It is not clear if such systems can work well on complex tasks (Lowe et al. 2017; He et al. 2018).”
τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (2024, background): “The latter either rely on user simulators that are rule-based or rely on symbolic specifications [17, 8] or perform tests with real humans through crowdsourcing platforms [9].”
Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models (2019, background): “or learned, to decouple the word generation process from dialog policy in E2E systems for better dialog policy (He et al., 2018; Yarats and Lewis, 2017).”
Adversarial Language Games for Advanced Natural Language Intelligence (2019, background): “(3) Negotiation requires agents to divide items with different values based on conversation (Sadri et al., 2001; Lewis et al., 2017; He et al., 2018).”
Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good (2019, background): “In order to achieve different negotiation goals, He et al. (2018) decoupled the dialogue act and language generation which helped control the strategy with more flexibility.”
Executing Instructions in Situated Collaborative Interactions (2019, background): “, 2017; Udagawa and Aizawa, 2019), and negotiation (Lewis et al., 2017; He et al., 2018).”
τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment (2025, unknown): “Following a long line of research into language agents [25, 27, 10, 13, 21], LLM tool use [24, 20, 9], and task-oriented dialog [5, 4, 1, 22, 6, 7, 8], τ -bench [26] is a recently introduced benchmark to measure the reliability of language agents in multi-turn task-oriented conversations such as…”
CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems (2021, unknown): “, 2017), and buyer-seller negotiations (He et al., 2018).”
Multi-agent deep reinforcement learning: a survey (2021, result): “Similar studies concerned with negotiations are covered in Cao et al. (2018) and He et al. (2018).”
Reward Design with Language Models (2023, background): “Agents negotiate for T time steps by exchanging coarse dialogue acts x t at each time step 1 ≤ t ≤ T (He et al., 2018).”
Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog (2019, background): “…(RL) is commonly used to learn dialog policy (Fatemi et al., 2016; Peng et al., 2017; Chen et al., 2017; Yarats and Lewis, 2018; Lei et al., 2018; He et al., 2018; Su et al., 2018), where users are modeled as a part of the environment and the policy is learned through interactions with users.”
SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents (2023, methodology): “Do not change the original meaning of the sentence. where <role> could be a “buyer” or a “seller”, the buyer’s target <price> is from CraigslistBargins (He et al., 2018), and the seller’s <price> is generated by first sample a markup ratio from an exponential distribution with rate parameter 0.5, th”