Preference-based interactive multi-document summarisation

By Yang Gao, Christian M. Meyer, Iryna Gurevych. 2019

28 citations recorded by the source corpus.

Interactive NLP is a promising paradigm to close the gap between automatic NLP systems and the human upper bound. Preference-based interactive learning has been successfully applied, but the existing methods require several thousand interaction rounds even in simulations with perfect user feedback. In this paper, we study preference-based interactive summarisation. To reduce the number of interaction rounds, we propose the Active Preference-based ReInforcement Learning (APRIL) framework. APRIL uses active learning to query the user, preference learning to learn a summary ranking function from the preferences, and neural Reinforcement learning to efficiently search for the (near-)optimal summary. Our results show that users can easily provide reliable preferences over summaries and that APRIL outperforms the state-of-the-art preference-based interactive method in both simulation and real-user experiments.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Iryna Gurevych
2. Luke S. Zettlemoyer
3. Mohit Bansal
4. Percy Liang
5. Jianfeng Gao
6. Yang Gao
7. Yejin Choi
8. Geoffrey Irving

In Natural Language Processing

1. Iryna Gurevych
2. Yang Gao
3. Wei Zhao
4. Steffen Eger
5. Christian M. Meyer
6. Ido Dagan
7. Alec Radford
8. Ori Shapira

Citations in the corpus

• Fine-Tuning Language Models from Human Preferences (2019, methodology): “…such as BLEU for translation (Ranzato et al., 2015; Wu et al., 2016), ROUGE for summarization (Ranzato et al., 2015; Paulus et al., 2017; Wu and Hu, 2018; Gao et al., 2019b), music theory-based rewards (Jaques et al., 2017), or event detectors for story generation (Tambwekar et al., 2018).”
• Better Rewards Yield Better Summaries: Learning to Summarise Without References (2019, methodology): “Note that ExtAbsRL is a cross-input RL while NeuralTD is an input-specific RL (see § 2).”
• SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization (2020, methodology): “Besides SCUs, recent work has explored eliciting preferences over summaries (Zopf, 2018; Gao et al., 2018, 2019a) and annotations of important bi-grams (P.V.S and Meyer, 2017) to derive summary ratings.”