RouteLLM: Learning to Route LLMs with Preference Data

By Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M. W. Kadous, Ion Stoica. arXiv.org, 2024

517 citations recorded by the source corpus.

Large language models (LLMs) exhibit impressive capabilities across a wide range of tasks, yet the choice of which model to use often involves a trade-off between performance and cost. More powerful models, though effective, come with higher expenses, while less capable models are more cost-effective. To address this dilemma, we propose several efficient router models that dynamically select between a stronger and a weaker LLM during inference, aiming to optimize the balance between cost and response quality. We develop a training framework for these routers leveraging human preference data and data augmentation techniques to enhance performance. Our evaluation on widely-recognized benchmarks shows that our approach significantly reduces costs-by over 2 times in certain cases-without compromising the quality of responses. Interestingly, our router models also demonstrate significant transfer learning capabilities, maintaining their performance even when the strong and weak models are changed at test time. This highlights the potential of these routers to provide a cost-effective yet high-performance solution for deploying LLMs.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/9b3239cff17327960804098e33e1ca903e7b9e85

Citations in the corpus

• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling (2024, unknown): “Additionally, multiple models can be ensembled together at inference time to combine their strengths [58, 14, 45, 56, 31].”
• From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge (2024, unknown): “…et al., 2023), (Sha et al., 2023), (Hao et al., 2023), (Zhou et al., 2024b), (Lahoti et al., 2023), (Liang et al., 2023), (Li et al., 2024b), (Besta et al., 2024), (Ong et al., 2024), (Zhao et al., 2024b), (Kawabata and Sugawara, 2024), (Xie et al., 2024b), (Lightman et al., 2023), (Li et al.)”
• Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models (2025, unknown): “RouteLLM [63] trains a query router to dispatch incoming queries to suitable LLMs based on complexity.”
• Harnessing the Wisdom of LLM Crowds through Complementarity-Driven Iterative Collaboration (2026, unknown): “RouteLLM (Ong et al. 2025) is designed to route between a strong but expensive model and a weak but cheap model to optimize the cost-quality tradeoff, an objective that does not apply to our setting where all workers are open-source models of comparable scale with no meaningful cost differential.”