Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models

By Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, Minjoon Seo. Conference on Empirical Methods in Natural Language Processing, 2024

471 citations recorded by the source corpus.

Proprietary LMs such as GPT-4 are often employed to assess the quality of responses from various LMs. However, concerns including transparency, controllability, and affordability strongly motivate the development of open-source LMs specialized in evaluations. On the other hand, existing open evaluator LMs exhibit critical shortcomings: 1) they issue scores that significantly diverge from those assigned by humans, and 2) they lack the flexibility to perform both direct assessment and pairwise ranking, the two most prevalent forms of assessment. Additionally, they do not possess the ability to evaluate based on custom evaluation criteria, focusing instead on general attributes like helpfulness and harmlessness. To address these issues, we introduce Prometheus 2, a more powerful evaluator LM than its predecessor that closely mirrors human and GPT-4 judgements. Moreover, it is capable of processing both direct assessment and pair-wise ranking formats grouped with a user-defined evaluation criteria. On four direct assessment benchmarks and four pairwise ranking benchmarks, Prometheus 2 scores the highest correlation and agreement with humans and proprietary LM judges among all tested open evaluator LMs. Our models, code, and data are all publicly available.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/ecdd53eaab7455daea27609b07a418a21aa7ad35

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild (2024, background): “…into three sections: the first section provides the high-level instruction, the task to be tested, and two model outputs; the second section specifies the checklist and the rules; and the last section instructs the LLM judge to follow the step-by-step evaluation process as detailed in Section 3.2”
ASAG2024: A Combined Benchmark for Short Answer Grading (2024, unknown): “We selecttwospecializedgrading systems, BART-SAF [8, 9] & PrometheusII-7B [7], that are employing task-ﬁnetuned language models.”
Understanding Generative AI Content with Embedding Models (2024, background): “Current techniques for this may not be entirely reliable; they depend on expensive benchmarks and instance-based evaluation which often requires an LLM judge that is prone to hallucinations and other biases [49, 61, 62].”