Multilingual Summarization Evaluation without Human Models

By Horacio Saggion, Juan-Manuel Torres-Moreno, Iria da Cunha, Eric SanJuan, Patricia Velázquez-Morales. 2010

97 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, background): “Saggion et al. (2010) first demonstrated that distributional differences between references and model-outputs can be used as a scoring mechanism.”
Better Metrics to Automatically Predict the Quality of a Text Summary (2012, unknown): “In [14], Saggion et al . expand on the work of Louis and Nenkova [15] by employing Jensen–Shannon divergence to contrast the probability distribution of a summary with that of its reference text.”
Multi-document multilingual summarization corpus preparation, Part 1: Arabic, English, Greek, Chinese, Romanian (2013, methodology): “While commonly used methods build upon a few human summaries to be able to judge automatic summaries (e.g., (Lin, 2004; Hovy et al., 2005)), there also exist works on fully automatic evaluation of summaries, without human “ model” summaries (Louis and Nenkova, 2012; Saggion et al., 2010).”
Fear the REAPER: A System for Automatic Multi-Document Summarization with Reinforcement Learning (2014, methodology): “We further explore the reward space by introducing similarity metric calculations used in ROUGE (Lin, 2004) and base our ideas on Saggion et al. (2010).”
Automatic Text Summarization: Past, Present and Future (2013, methodology): “Finally, [68] showed that this method is effective in most cases but not always: they specifically pointed out that the method does not perform so well for biographical information or for the summarization of opinions.”
Multi-document multilingual summarization corpus preparation, Part 2: Czech, Hebrew and Spanish (2013, background): “A second document (Elhadad et al., 2013) elaborates on the rest of the languages.”
Multi-document multilingual summarization and evaluation tracks in ACL 2013 MultiLing Workshop (2013, methodology): “, 2005)), there also exist works on fully automatic evaluation of summaries, without human“model” summaries (Louis and Nenkova, 2012; Saggion et al., 2010).”
System Combination for Multi-document Summarization (2015, background): “The most closely related papers are the ones that investigated automatic evaluation of summarization without human references (Louis and Nenkova, 2009; Saggion et al., 2010), where the effectiveness”