Transforming Wikipedia Into Augmented Data for Query-Focused Summarization
By Haichao Zhu, Li Dong, Furu Wei, Bing Qin, Ting-Wei Liu. 2019
25 citations recorded by the source corpus.
The limited size of existing query-focused summarization datasets renders training data-driven summarization models challenging. Meanwhile, the manual construction of a query-focused summarization corpus is costly and time-consuming. In this paper, we use Wikipedia to automatically collect a large query-focused summarization dataset (named WikiRef) of more than 280,000 examples, which can serve as a means of data augmentation. We also develop a BERT-based query-focused summarization model (Q-BERT) to extract sentences from the documents as summaries. To better adapt a huge model containing millions of parameters to tiny benchmarks, we identify and fine-tune only a sparse subnetwork, which corresponds to a small fraction of the whole model parameters. Experimental results on three DUC benchmarks show that the model pre-trained on WikiRef has already achieved reasonable performance. After fine-tuning on the specific benchmark datasets, the model with data augmentation outperforms strong comparison systems. Moreover, both our proposed Q-BERT model and subnetwork fine-tuning further improve the model performance.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Summarization
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Xipeng Qiu
- Yang Liu
- Dragomir R. Radev
- Asli Celikyilmaz
- Jianfeng Gao
- Shafiq R. Joty
- Alexander R. Fabbri
- Percy Liang
In Computer Vision and Pattern Recognition
- Yang Liu
- Marjan Ghazvininejad
- Haoran Li
- Da Yin
- Ming Zhong
- Asli Celikyilmaz
- Xipeng Qiu
- Ahmed Hassan Awadallah
In Summarization
Citations in the corpus
- Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation (2020, methodology): “In other work, Lebanoff et al. (2018) adapt a single-document summarization model to the multi-document setting, while Zhu et al. (2019) use the references from Wikipedia data for downstream query-based summarization, similar to the task of Wikipedia paragraph generation as defined in Liu et al.…”
- QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization (2021, background): “However, the models focus on news (Dang, 2005, 2006), debate (Nema et al., 2017), and Wikipedia (Zhu et al., 2019).”