Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization

By Shashi Narayan, Shay B. Cohen, Mirella Lapata. 2018

2,107 citations recorded by the source corpus.

We introduce “extreme summarization”, a new single-document summarization task which does not favor extractive strategies and calls for an abstractive modeling approach. The idea is to create a short, one-sentence news summary answering the question “What is the article about?”. We collect a real-world, large-scale dataset for this task by harvesting online articles from the British Broadcasting Corporation (BBC). We propose a novel abstractive model which is conditioned on the article’s topics and based entirely on convolutional neural networks. We demonstrate experimentally that this architecture captures long-range dependencies in a document and recognizes pertinent content, outperforming an oracle extractive system and state-of-the-art abstractive approaches when evaluated automatically and by humans.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

PaLM: Scaling Language Modeling with Pathways (2022, background): “[en/es/ru/tr/vi → en] • XSum (Narayan et al., 2018) – Summarize a news article in a single sentence.”
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (2019, methodology): “In contrast, XSum is highly abstractive, and extractive models perform poorly.”
SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders (2019, background): “…weather to be the coolest and fun day and he went to the beach and got ready. primarily from the news domain, abstractive summarization has been approached as a supervised neural sequence transduction problem (Rush et al., 2015; Nallapati et al., 2016; Narayan et al., 2018; Fabbri et al., 2019).”
fairseq: A Fast, Extensible Toolkit for Sequence Modeling (2019, background): “…2019), language modeling (Dauphin et al., 2017; Baevski and Auli, 2019), abstractive document summarization (Fan et al., 2018a; Liu et al., 2018; Narayan et al., 2018), story generation (Fan et al., 2018b, 2019), error correction (Chollampatt and Ng, 2018), multilingual sentence embeddings…”
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity (2021, methodology): “, 2015) and BBC XSum (Narayan et al., 2018) data sets are used to measure the ability to summarize articles.”
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text (2022, methodology): “These findings has also been corroborated for XSum (Gehrmann et al., 2021; Mille et al., 2021).”
Finetuned Language Models Are Zero-Shot Learners (2021, neutral): “XSum (Narayan et al., 2018) 8.”
ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models (2021, background): “XSum (Narayan et al., 2018) is an abstractive summarization task requiring models to summarize a news article in a single sentence.”
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus (2021, methodology): “We examine contamination of target text in test sets of datasets for three generation tasks: (i) abstractive summarization (TIFU, Kim et al., 2019; XSum, Narayan et al., 2018), (ii) table-to-text generation (WikiBio, Lebret et al., 2016), and (iii) graph-to-text generation (AMR-to-text, LDC2017T10).”
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics (2021, background): “…et al., 2020) Produce an English sentence that describes the highlighted cells in the context of the given table. en 136k Highlighted Table XSum (Narayan et al., 2018) Highlight relevant points in a news article en *25k Articles WebNLG (Gardent et al., 2017) Produce a text that verbalises the…”
WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization (2020, background): “The majority of research in abstractive summarization has focused on monolingual summarization in English (Gehrmann et al., 2018; Song et al., 2020; Narayan et al., 2018).”
Big Bird: Transformers for Longer Sequences (2020, neutral): “Instances Input Length Output Length Dataset Training Dev Test Median 90%-ile Median 90%-ile BBC XSum [68] 204044 11332 11334 359 920 25 32 CNN/DailyMail [37] 287113 13368 11490 777 1439 59 93”
MLSUM: The Multilingual Summarization Corpus (2020, methodology): “A similar approach has been adopted for several dataset releases in the recent past, such as Question Answering Corpus (Hermann et al., 2015b) or XSUM (Narayan et al., 2018a).”
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization (2019, methodology): “In recent years, there has been increased interest in collecting new summarization datasets that have more abstractive summaries (Narayan et al., 2018), have longer documents, (Cohan et al.”
Hindsight: Posterior-guided training of retrievers for improved open-ended generation (2021, background): “Many more generation tasks such as summarization (Narayan et al., 2018) and story generation (Mostafazadeh et al.”
Few-Shot Text Generation with Natural Language Instructions (2020, background): “• XSum (Narayan et al., 2018): Summarize news articles spanning a wide range of different topics.”
Prefix-Tuning: Optimizing Continuous Prompts for Generation (2021, methodology): “For the summarization task, we use the XSUM (Narayan et al., 2018) dataset, which is an abstractive summarization dataset on news articles.”
CM3: A Causal Masked Multimodal Model of the Internet (2022, background): “XSum (Narayan et al., 2018) provides abstractive single sentence summaries of news articles.”