Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
By Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Chen and 2 more. International Conference on Learning Representations, 2024
366 citations recorded by the source corpus.
LLM-as-a-Judge has been widely utilized as an evaluation method in various benchmarks and served as supervised rewards in model training. However, despite their excellence in many domains, potential issues are under-explored, undermining their reliability and the scope of their utility. Therefore, we identify 12 key potential biases and propose a new automated bias quantification framework-CALM-which systematically quantifies and analyzes each type of bias in LLM-as-a-Judge by using automated and principle-guided modification. Our experiments cover multiple popular language models, and the results indicate that while advanced models have achieved commendable overall performance, significant biases persist in certain specific tasks. Empirical results suggest that there remains room for improvement in the reliability of LLM-as-a-Judge. Moreover, we also discuss the explicit and implicit influence of these biases and give some suggestions for the reliable application of LLM-as-a-Judge. Our work highlights the need for stakeholders to address these issues and remind users to exercise caution in LLM-as-a-Judge applications.
Subfields: Economics and Econometrics, Law, Strategy and Management
View Paper: https://www.semanticscholar.org/paper/e142887ab6516654f25e233a3e661eddac123630
Citations in the corpus
- LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks (2024, neutral): “Additionally, some datasets adopt LLM-as-a-judge (Zheng et al., 2023; Li et al., 2024a) for evaluation, which can be costly and may introduce biases in their assessments (Bai et al., 2024c; Wang et al., 2024b; Ye et al., 2025).”
- From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge (2024, neutral): “Typical benchmarks include EVALBIAS-BENCH (Wang et al., 2024c) and CALM (Ye et al., 2024a), focus explicitly on quantifying biases, including those emerging from alignment and robustness under adversarial conditions.”
- HealthBench: Evaluating Large Language Models Towards Improved Human Health (2025, neutral): “Length bias is a known issue of evaluations that rely on model-based graders (Ye et al., 2024; Chen et al., 2024).”