Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate

By Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, Shuming Shi. Conference on Empirical Methods in Natural Language Processing, 2023

1,247 citations recorded by the source corpus.

Modern large language models (LLMs) like ChatGPT have shown remarkable performance on general language tasks but still struggle on complex reasoning tasks, which drives the research on cognitive behaviors of LLMs to explore human-like problem-solving strategies. Along this direction, one representative strategy is self-reflection, which asks an LLM to refine the solution with the feedback generated by itself iteratively. However, our study shows that such reflection-style methods suffer from the Degeneration-of-Thought (DoT) problem: once the LLM has established confidence in its solutions, it is unable to generate novel thoughts later through reflection even if its initial stance is incorrect. To address the DoT problem, we propose a Multi-Agent Debate (MAD) framework, in which multiple agents express their arguments in the state of “tit for tat” and a judge manages the debate process to obtain a final solution. Clearly, our MAD framework encourages divergent thinking in LLMs which would be helpful for tasks that require deep levels of contemplation. Experiment results on two challenging datasets, commonsense machine translation and counter-intuitive arithmetic reasoning, demonstrate the effectiveness of our MAD framework. Extensive analyses suggest that the adaptive break of debate and the modest level of “tit for tat” state are required for MAD to obtain good performance. Moreover, we find that LLMs might not be a fair judge if different LLMs are used for agents.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/385c74957858e7d6856d48e72b5a902b4c1aa28c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Improving Factuality and Reasoning in Language Models through Multiagent Debate (2023, positive): “In Table A6, we find that our approach substantially outperforms Liang et al. (2023), as it more effectively encourages language models to focus on reasoning.”
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models (2023, background): “Recent strategies involve inference-time self-consistency checks (Manakul et al., 2023), multi-agent debating (Du et al., 2023; Liang et al., 2023), and inference-time intervention using human labels (Li et al., 2023).”
Large Language Models Cannot Self-Correct Reasoning Yet (2023, background): “For instance, Madaan et al. (2023) demonstrate the promising potential of self-correction across various tasks, yet mathematical reasoning shows negligible improvement.”
Cognitive Architectures for Language Agents (2023, methodology): “Language agents (Weng, 2023; Wang et al., 2023b; Xi et al., 2023; Yao and Narasimhan, 2023) are an emerging class of artifical intelligence (AI) systems that use large language models (LLMs; Vaswani et al., 2017; Brown et al., 2020; Devlin et al., 2019; OpenAI, 2023a) to interact with the world.”
MetaGPT: Meta Programming for Multi-Agent Collaborative Framework (2023, background): “However, existing systems [1, 2, 3, 4, 5, 6, 7] tend to oversimplify the complexities inherent to real-world applications.”
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge (2024, neutral): “In multi-agent collaboration systems, both Liang et al. (2023) and Li et al. (2024b) propose to leverage the judge LLM to select the most feasible and reasonable solutions among multiple candidates’ responses.”
Mixture-of-Agents Enhances Large Language Model Capabilities (2024, background): “Around the same time, MAD (Liang et al., 2023) introduces an asymmetric mechanism design, with different roles, i.e., debater and judge.”
Empowering Biomedical Discovery with AI Agents (2024, neutral): “Diﬀerent strategies are used for multi-agent interaction, including cooperation [129–131] and negotiation [73, 132, 133].”
Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs (2024, neutral): “There are also several contributions made to predict the distribution of the computation budget and allocate the computation power based on the prompt’s difficulty (Damani et al., 2024; Wang et al., 2024; Xu et al., 2024).”
MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning (2023, background): “Liang et al. (2023) proposed a multi-agent debate framework in which various agents put forward their statements in a tit for tat pattern.”
A survey on large language model based autonomous agents (2023, neutral): “In the MAD(Multi-Agent Debate) [83] framework, multiple agents express their arguments in an "eye-for-an-eye" fashion, and a judge manages the debate process to reach a final solution.”
Multi-Agent Collaboration Mechanisms: A Survey of LLMs (2025, positive): “Researchers have found taking multiple LLM instances to debate for a fixed number of rounds can boost their factuality and reasoning capabilities [41, 77, 140].”
The rise and potential of large language model based agents: a survey (2025, positive): “Furthermore, through iterations of interactions and debates between agents, individual errors such as hallucinations or degeneration of thought (DoT) are corrected by the group, ultimately improving performance on the task [92,93].”
ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs (2023, methodology): “Across seven benchmarks spanning commonsense reasoning, mathematical reasoning, logical reasoning, and Natural Language Inference (NLI), R E C ONCILE outperforms prior single-agent (e.g., Self-Refine (Madaan et al., 2023) and Self-consistency (Wang et al., 2023b)) and multi-agent baselines (Debate (”
ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving (2023, background): “Mathematical Reasoning Recent research has greatly improved reasoning in LLMs with step-by-step natural language reasoning (Wei et al., 2022; Zhou et al., 2023; Zhu et al., 2023; Huang et al., 2022; Liang et al., 2023).”
AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation (2023, positive): “• Multi-Agent Debate: Two recent works investigate and show that multi-agent debate is an effective way to encourage divergent thinking in LLMs (Liang et al., 2023) and to improve the factuality and reasoning of LLMs (Du et al., 2023).”