A Methodology for Extrinsic Evaluation of Text Summarization: Does ROUGE Correlate?
By Bonnie J. Dorr, Christof Monz, Stacy President, Richard M. Schwartz, David M. Zajic. 2005
44 citations recorded by the source corpus.
Subfields: Natural Language Processing, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Elizabeth Clark
- Thibault Sellam
- Sebastian Gehrmann
- Anya Belz
- Ehud Reiter
- Karen Spärck Jones
- Emiel Krahmer
- Albert Gatt
In Evaluation & Benchmarks
- Elizabeth Clark
- Sebastian Gehrmann
- Thibault Sellam
- Ehud Reiter
- Anya Belz
- Karen Spärck Jones
- Albert Gatt
- Ondrej Dusek
Citations in the corpus
- Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “For that reason, ROUGE underperforms on non-extractive summaries (Dorr et al., 2005).”
- Automatic summarising: The state of the art (2007, background): “Dorr et al. (2005) compared assessments on summaries with reference assessments of sources, i.e. gold-standard annotations on sources, which changes the people involved without proper control.”
- An Investigation into the Validity of Some Metrics for Automatically Evaluating Natural Language Generation Systems (2009, background): “Dorr et al. (2005) checked if ROUGE scores correlated with task effectiveness; they did not find a strong correlation.”