Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: A Preliminary Empirical Study

73 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Direct Preference Optimization: Your Language Model is Secretly a Reward Model (2023, methodology): “While existing studies suggest LMs can be better automated evaluators than existing metrics [10], we conduct a human study to justify our usage of GPT-4 for evaluation in Sec.”
Evaluating Large Language Models at Evaluating Instruction Following (2023, methodology): “Therefore, several works introduce meta-evaluation benchmarks, including FairEval (Wang et al., 2023b), MT-Bench (Zheng et al., 2023), and LLMEval 2 (Zhang et al., 2023), to examine whether LLM evaluators have high agreement with humans.”
Benchmarking Cognitive Biases in Large Language Models as Evaluators (2023, methodology): “For instance, Liu et al. (2023); Chen et al. (2023); Wang et al. (2023a) investigated the effectiveness of GPT-4 and ChatGPT against reference-free evaluation meth-ods, whereas Fu et al. (2023) proposed an evaluation framework, GPTS CORE , to score generated texts.”
Aligning Large Language Models with Human: A Survey (2023, methodology): “Differently, Chen et al. (2023b) propose using ChatGPT to directly assess the quality of instructions by assigning scores.”