Towards Evaluation in Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “This property was focused on by Dale and Mellish (1998) who position the evaluation of NLG systems as complementary to NLU systems, pointing out the symmetry of moving from natural language to representations of meaning or structure (NLU) or the other way around (NLG).”
A Snapshot of NLG Evaluation Practices 2005 - 2014 (2015, background): “Past work within the NLG community has looked at the issues of evaluating NLG techniques and systems, the challenges unique to the NLG context in comparison to Natural Language Analysis (Dale and Mellish, 1998), and the comparisons between evaluation approaches (Belz and Reiter, 2006).”
Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation (2019, background): “the goodness of texts” (Dale and Mellish, 1998), and there lacks a clear model of text quality (Hard-”
RoMe: A Robust Metric for Evaluating Natural Language Generation (2022, background): “Evaluating generative systems such as text summarization, dialogue systems, and machine translation is challenging since the assessment involves several criteria such as content determination, lexicalization, and surface realization (Liu et al., 2016; Dale and Mellish, 1998).”