Is ChatGPT a Good NLG Evaluator? A Preliminary Study

By Jiaan Wang, Yunlong Liang, Fandong Meng, Zengkui Sun, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, Jie Zhou. NEWSUM, 2023

643 citations recorded by the source corpus.

Recently, the emergence of ChatGPT has attracted wide attention from the computational linguistics community. Many prior studies have shown that ChatGPT achieves remarkable performance on various NLP tasks in terms of automatic evaluation metrics. However, the ability of ChatGPT to serve as an evaluation metric is still underexplored. Considering assessing the quality of natural language generation (NLG) models is an arduous task and NLG metrics notoriously show their poor correlation with human judgments, we wonder whether ChatGPT is a good NLG evaluation metric. In this report, we provide a preliminary meta-evaluation on ChatGPT to show its reliability as an NLG metric. In detail, we regard ChatGPT as a human evaluator and give task-specific (e.g., summarization) and aspect-specific (e.g., relevance) instruction to prompt ChatGPT to evaluate the generated results of NLG models. We conduct experiments on five NLG meta-evaluation datasets (including summarization, story generation and data-to-text tasks). Experimental results show that compared with previous automatic metrics, ChatGPT achieves state-of-the-art or competitive correlation with human judgments in most cases. In addition, we find that the effectiveness of the ChatGPT evaluator might be influenced by the creation method of the meta-evaluation datasets. For the meta-evaluation datasets which are created greatly depending on the reference and thus are biased, the ChatGPT evaluator might lose its effectiveness. We hope our preliminary study could prompt the emergence of a general-purposed reliable NLG metric.

Subfields: Artificial Intelligence, Health Informatics

View Paper: https://www.semanticscholar.org/paper/8221f1597000543432b7021ca79dbc51a7a63f9c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chenguang Zhu
2. Yang Liu
3. Yichong Xu
4. Xiang Lisa Li
5. Noah D. Goodman
6. Shuming Shi
7. Jesse Mu
8. Hung-yi Lee

In Health Informatics

1. Siyuan Ma
2. Yiheng Liu
3. Antong Li
4. Zheng Liu
5. Haoyang He
6. Mengshen He
7. Dajiang Zhu
8. Tianle Han

Citations in the corpus

• Learning to Compress Prompts with Gist Tokens (2023, methodology): “Recent work has found that ChatGPT can be used for text annotation and evaluation (Gilardi et al., 2023; Huang et al., 2023; Wang et al., 2023) with comparable performance to human annotation, and similar model-based evaluations have been conducted with recent LMs (Taori et al.”
• Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: A Preliminary Empirical Study (2023, background): “Recent studies (Fu et al., 2023; Wang et al., 2023; Kocmi and Federmann, 2023; Ji et al., 2023) have also demonstrated the potential of LLMs in evaluating the quality of generated texts.”
• Can Large Language Models Be an Alternative to Human Evaluations? (2023, methodology): “During the reviewing process, we found some works on arXiv (Wang et al., 2023; Huang et al., 2023; Gilardi et al., 2023) that explore a similar idea using different tasks.”
• Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond (2023, background): “Some other evaluators based on LLMs [37, 60, 74, 118] also show good human alignment in more NLG tasks, especially compared with traditional metrics.”
• ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate (2023, background): “Wang et al. (2023a) explore the potential of utilizing ChatGPT as an NLG evaluator by prompting it to score a text directly.”
• Benchmarking Cognitive Biases in Large Language Models as Evaluators (2023, background): “As we are looking to test the ability of language models to perform as unbiased evaluators to judge response quality and correctness, the Q/A response format presents the most natural setting for these comparisons.”
• ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems (2023, neutral): “Other work has used LLM prompting to evaluate system quality across natural language generation tasks, such as translation, summarization, and dialogue generation (Kocmi and Federmann, 2023; Fu et al., 2023; Liu et al., 2023a; Wang et al., 2023).”
• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, neutral): “Recently, the remarkable instruction-following ability of LLMs has underscored their potential for automatic evaluation (Chiang and Lee, 2023; Liu et al., 2023g; Wang et al., 2023d).”
• From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge (2024, neutral): “Although these small model-based metrics shift from word-level to embedding-level representations and offer greater flexibility, they still struggle to capture subtle attributes like help-fulness and harmlessness beyond relevance.”
• Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models (2023, background): “Wang et al. [93] used ChatGPT as a natural language generation (NLG) evaluator to study the correlation with human judgment.”
• G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment (2023, methodology): “Recent studies propose directly using LLMs as reference-free NLG evaluators (Fu et al., 2023; Wang et al., 2023a).”
• Summary of ChatGPT/GPT-4 Research and Perspective Towards the Future of Large Language Models (2023, methodology): “[72] used ChatGPT as a natural language generation (NLG) evaluator to study the correlation with human judgment.”
• GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts (2023, background): “• APIs and ChatGPT Assistance: Utilizing content moderation APIs [59] or enlisting ChatGPT for labeling assistance [35,53,61] are other proposed solutions.”
• ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning (2023, background): “…line of work concerns the performance comparison of ChatGPT and state-of-the-art systems for important tasks in NLP such as text summarization (Wang et al., 2023a; Yang et al., 2023), machine translation (Hendy et al., 2023; Jiao et al., 2023; Kocmi and Federmann, 2023), question answering…”
• RAGAs: Automated Evaluation of Retrieval Augmented Generation (2023, background): “Another approach directly asks ChatGPT to evaluate a particular aspect of the given answer by providing a score between 0 and 100, or by providing a rating on a 5-star scale (Wang et al., 2023a).”
• Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving (2023, background): “5 to grade our model’s answers which is a recently emerging technique grading natural language responses [46] [47] [48].”