Searching for Effective Neural Extractive Summarization: What Works and What’s Next

By Ming Zhong, Pengfei Liu, Danqing Wang, Xipeng Qiu, Xuanjing Huang. 2019

161 citations recorded by the source corpus.

The recent years have seen remarkable success in the use of deep neural networks on text summarization. However, there is no clear understanding of why they perform so well, or how they might be improved. In this paper, we seek to better understand how neural extractive summarization systems could benefit from different types of model architectures, transferable knowledge and learning schemas. Besides, we find an effective way to improve the current framework and achieve the state-of-the-art result on CNN/DailyMail by a large margin based on our observations and analysis. Hopefully, our work could provide more hints for future research on extractive summarization.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xipeng Qiu
2. Mohit Bansal
3. Pengfei Liu
4. Jianfeng Gao
5. Zhe Gan
6. Jingjing Liu
7. Yang Liu
8. Xuanjing Huang

In Natural Language Processing

1. Xipeng Qiu
2. Pengfei Liu
3. Peng Liu
4. Dragomir R. Radev
5. Yang Liu
6. Xuanjing Huang
7. Mohit Bansal
8. Jianfeng Gao

Citations in the corpus

• PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization (2019, background): “Lead had decent performance on the two news datasets but was significantly worse on the two non-news datasets, which agrees findings of lead bias in news datasets (See et al., 2017; Zhong et al., 2019).”
• Pre-trained models for natural language processing: A survey (2020, neutral): “[218] introduced transferable knowledge (e.”
• BARTScore: Evaluating Generated Text as Text Generation (2021, neutral): “Extractive Systems E1: BanditSum [11]; E2: Refresh [49]; E3: NeuSum [81]; E4: LSTM-PN-RL [80]; E5: BERT-TF-SL [80]; E6: BERT-TF-PN [80]; E7: BERT-LSTM-PN-RL [80]; E8: BERT-LSTM-PN [80]; E9: HeterGraph [69]; E10: MatchSum [79].”
• Automatic Text Evaluation through the Lens of Wasserstein Barycenters (2021, background): “, 2017) and 14 abstractive systems (Zhong et al., 2020; Wang et al., 2020; Zhong et al., 2019; Liu and Lapata, 2019; Zhou et al., 2018; Narayan et al., 2018; Dong et al., 2019; Kedzie et al., 2018; Zhou et al., 2018).”
• Heterogeneous Graph Neural Networks for Extractive Document Summarization (2020, methodology): “Recently, pre-trained language models are also applied in summarization for contextual word representations (Zhong et al., 2019a; Liu and Lapata, 2019b; Xu et al., 2019; Zhong et al., 2020).”
• Are Factuality Checkers Reliable? Adversarial Meta-evaluation of Factuality in Summarization (2021, methodology): “With the rapid development of neural networks in text summarization (Liu and Lapata, 2019; Liu, 2019; Zhong et al., 2019; Zhang et al., 2019; Lewis et al., 2019; Zhong et al., 2020; Liu and Liu, 2021), especially the use of contextualized pre-trained models (Devlin et al., 2019; Lewis et al.,…”
• How well do you know your summarization datasets? (2021, unknown): “Coverage , Density, Redundancy Grusky et al. (2018); Bommasani and Cardie (2020); Zhong et al. (2019b) use similar forms of token-level coverage between the source and the reference to measure the extractiveness of the summary.”
• RefSum: Refactoring Neural Summarization (2021, unknown): “6 In details, for §5.7 we use the following systems: pointer-generator+coverage (See et al., 2017), REFRESH (Narayan et al., 2018b), fastAbsRL-rank (Chen and Bansal, 2018), CNN-LSTM-BiClassifier (Kedzie et al., 2018), CNN-Transformer-BiClassifier (Zhong et al., 2019), CNN-Transformer-Pointer (Zhong et”
• QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization (2021, methodology): “…summarization (Rush et al., 2015; Chopra et al., 2016; Nallapati et al., 2016; See et al., 2017; Celikyilmaz et al., 2018; Chen and Bansal, 2018; Zhong et al., 2019a; Xu and Durrett, 2019; Liu and Lapata, 2019; Lebanoff et al., 2019; Cho et al., 2019; Zhong et al., 2020; Wang et al., 2020; Xu…”
• Towards a Unified Multi-Dimensional Evaluator for Text Generation (2022, background): “The motivation is that the first few sentences in the news tend to be salient and informative (See et al., 2017; Zhong et al., 2019), so it allows the model to learn inter-sentence coherence while also capturing the central idea of the document.”
• Discourse-Aware Neural Extractive Text Summarization (2019, methodology): “PNBert proposed a BERT-based model with various training strategies, including reinforcement learning and Pointer Networks (Zhong et al., 2019).”
• Extractive Summarization as Text Matching (2020, methodology): “B ANDIT S UM (Dong et al., 2018) 41.50 18.70 37.60 N EU S UM (Zhou et al., 2018) 41.59 19.01 37.98 J ECS (Xu and Durrett, 2019) 41.70 18.50 37.90 H I B ERT (Zhang et al., 2019b) 42.37 19.95 38.83 P N B ERT (Zhong et al., 2019a) 42 0 .”
• Re-evaluating Evaluation in Text Summarization (2020, unknown): “…al., 2019b), Bert-Sum-Ext (Liu and Lapata, 2019a), CNN-Transformer-BiClassifier (CTrans-SL; Zhong et al. (2019)), CNN-Transformer-Pointer (CTrans-PN; Zhong et al. (2019)), HeterGraph (Wang et al., 2020) and MatchSum (Zhong et al., 2020) as representatives of extractive systems, totaling 11…”