LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion

By Dongfu Jiang, Xiang Ren, Bill Yuchen Lin. Annual Meeting of the Association for Computational Linguistics, 2023

639 citations recorded by the source corpus.

We present LLM-Blender, an ensembling framework designed to attain consistently superior performance by leveraging the diverse strengths of multiple open-source large language models (LLMs). Our framework consists of two modules: PairRanker and GenFuser, addressing the observation that optimal LLMs for different examples can significantly vary. PairRanker employs a specialized pairwise comparison method to distinguish subtle differences between candidate outputs. It jointly encodes the input text and a pair of candidates, using cross-attention encoders to determine the superior one. Our results demonstrate that PairRanker exhibits the highest correlation with ChatGPT-based ranking. Then, GenFuser aims to merge the top-ranked candidates, generating an improved output by capitalizing on their strengths and mitigating their weaknesses. To facilitate large-scale evaluation, we introduce a benchmark dataset, MixInstruct, which is a mixture of multiple instruction datasets featuring oracle pairwise comparisons. Our LLM-Blender significantly outperform individual LLMs and baseline methods across various metrics, establishing a substantial performance gap.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/993df7df129f8d18816877d69923d7df7b347d85

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

SimPO: Simple Preference Optimization with a Reference-Free Reward (2024, positive): “We then use llm-blender/PairRM [41] to score the 5 responses, selecting the highest-scoring one as y w and the lowest-scoring one as y l .”
Mixture-of-Agents Enhances Large Language Model Capabilities (2024, methodology): “In order to better leverage the responses of multiple models, Jiang et al. (2023) trained a G EN F USER , a model that was trained to generate an improved response to capitalize on the strengths of multiple candidates.”
Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs (2024, background): “Existing research (Dong et al., 2024; Jiang et al., 2023; Touvron et al., 2023) frequently leverages a mixture of preference datasets from multiple sources to train reward models.”
Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models (2024, neutral): “4B (Jiang et al., 2023c) (pair-wise ranking).”
ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs (2023, methodology): “Jiang et al. (2023) propose LLM-Blender, a method to rank and fuse generations from different models.”
RouteLLM: Learning to Route LLMs with Preference Data (2024, neutral): “LLM-BLENDER [18] employs an ensemble framework that calls multiple LLMs at inference and uses a router model to select the best response.”
Large Language Monkeys: Scaling Inference Compute with Repeated Sampling (2024, positive): “Additionally, multiple models can be ensembled together at inference time to combine their strengths [58, 14, 45, 56, 31].”
Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing (2024, methodology): “GPT-ranking (Jiang et al., 2023), as a representative example, employs GPT models (e.g., GPT-4 (OpenAI, 2023)) to provide relative rankings between pairs of generated outputs.”
Multi-Agent Collaboration Mechanisms: A Survey of LLMs (2025, neutral): “[64] SOT NLG • Generate the skeleton of answer.”
Harnessing the Wisdom of LLM Crowds through Complementarity-Driven Iterative Collaboration (2026, unknown): “This fusion model is fine-tuned from a pre-trained LLM that takes the input query and K responses and produces an improved output as the final response (Jiang et al. 2023).”
S-SPPO: Semantic-Calibrated Self-Play Preference Optimization (2026, unknown): “To estimate the winning probability distribution (cid:98) P ( y i ≻ π t | x ) , we calculate the average win rate of a response y i against all other sampled responses y k : The pairwise probability P ( y ≻ y ′ | x ) is derived from the PairRM scoring function s ( y , y ′ ; x ) , which reflects the ”
Are Smaller Open-Weight LLMs Closing the Gap to Proprietary Models for Biomedical Question Answering? (2025, unknown): “As emphasized by Jiang et al. [31], different LLMs, trained on varied data and architectures, inherently exhibit unique strengths that can be synergistic in an ensemble.”