Objective Function Learning to Match Human Judgements for Optimization-Based Summarization

By Maxime Peyrard, Iryna Gurevych. 2018

20 citations recorded by the source corpus.

Supervised summarization systems usually rely on supervision at the sentence or n-gram level provided by automatic metrics like ROUGE, which act as noisy proxies for human judgments. In this work, we learn a summary-level scoring function \theta including human judgments as supervision and automatically generated data as regularization. We extract summaries with a genetic algorithm using \theta as a fitness function. We observe strong and promising performances across datasets in both automatic and manual evaluation.

Subfields: Artificial Intelligence, Natural Language Processing, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Optimization & Training

Citations in the corpus

Better Rewards Yield Better Summaries: Learning to Summarise Without References (2019, methodology): “In § 6, we show that our reward model does not use reference summaries but outperforms the feature-based baseline by Peyrard and Gurevych (2018) as well as ROUGE.”
CLIPScore: A Reference-free Evaluation Metric for Image Captioning (2021, background): “…to the machine translation cases highlighted in the introduction, reference-free evaluations have been proposed for other generation tasks, including summarization (Louis and Nenkova, 2013; Peyrard and Gurevych, 2018; Sun and Nenkova, 2019) and dialogue (Tao et al., 2018; Mehri and Eskenazi, 2020).”
Reward Learning for Efficient Reinforcement Learning in Extractive Document Summarisation (2019, background): “• TFIDF values averaged over all tokens and named entities in y [ Peyrard and Gurevych, 2018 ] .”
Studying Summarization Evaluation Metrics in the Appropriate Scoring Range (2019, methodology): “The SampleSummaries function is the genetic algorithm introduced genetic algorithm for summarization (Peyrard and Eckle-Kohler, 2016; Peyrard and Gurevych, 2018).”
HighRES: Highlight-based Reference-less Evaluation of Summarization (2019, background): “Evaluation Metrics Despite differences in the exact definitions, the majority (e.g., Hsu et al., 2018; Celikyilmaz et al., 2018; Narayan et al., 2018b; Chen and Bansal, 2018; Peyrard and Gurevych, 2018) agree on both or either one of two broad quality definitions: coverage determines how much of the”