BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization

By Eva Sharma, Chen Li, Lu Wang. 2019

272 citations recorded by the source corpus.

Most existing text summarization datasets are compiled from the news domain, where summaries have a flattened discourse structure. In such datasets, summary-worthy content often appears in the beginning of input articles. Moreover, large segments from input articles are present verbatim in their respective summaries. These issues impede the learning and evaluation of systems that can understand an article’s global content structure as well as produce abstractive summaries with high compression ratio. In this work, we present a novel dataset, BIGPATENT, consisting of 1.3 million records of U.S. patent documents along with human written abstractive summaries. Compared to existing summarization datasets, BIGPATENT has the following properties: i) summaries contain a richer discourse structure with more recurring entities, ii) salient content is evenly distributed in the input, and iii) lesser and shorter extractive fragments are present in the summaries. Finally, we train and evaluate baselines and popular learning models on BIGPATENT to shed light on new challenges and motivate future directions for summarization research.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics (2021, neutral): “Big Patent (Sharma et al., 2019) 5.”
Big Bird: Transformers for Longer Sequences (2020, background): “The reasons are two fold: First, the salient content can be evenly distributed in the long document, not just in ﬁrst 512 tokens, and this is by design in the BigPatents dataset [78].”
MLSUM: The Multilingual Summarization Corpus (2020, background): “Sharma et al. (2019), among others, showed that this is a robust baseline for several summarization datasets such as CNN/DM, NYT and BIGPATENT.”
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization (2019, background): “, 2018), have longer documents, (Cohan et al., 2018; Sharma et al., 2019), utilize multiple documents (Fabbri et al.”
Recursively Summarizing Books with Human Feedback (2021, background): “…Collins et al., 2017; Subramanian et al., 2019; Cohan et al., 2018; Xiao and Carenini, 2019; Zhao et al., 2020; Sotudeh et al., 2020), and patents (Sharma et al., 2019), as well as multi-document summarization (Liu et al., 2018; Ma et al., 2020; Gharebagh et al., 2020; Chandrasekaran et al.,…”
Muppet: Massive Multi-task Representations with Pre-Finetuning (2021, methodology): “, 2018), (Sharma et al., 2019) for summarization, and (3) Chunking, Constituency Parsing and Part-Of-Speech tagging for structured prediction from the Penn Treebank dataset (Marcus et al.”
Evaluating the Factual Consistency of Abstractive Text Summarization (2019, background): “…learning (Mc-Cann et al., 2017; Peters et al., 2018; Devlin et al., 2018), and availability of large-scale supervised datasets (Sandhaus, 2008; Nallapati et al., 2016; Grusky et al., 2018; Narayan et al., 2018; Sharma et al., 2019) allowed deep learning-based approaches to dominate the ﬁeld.”
On Extractive and Abstractive Neural Document Summarization with Transformer Language Models (2020, methodology): “Datasets We experiment with four different large-scale and long document summarization datasets - arXiv, PubMed (Cohan et al., 2018), bigPatent (Sharma et al., 2019) and Newsroom (Grusky et al., 2018a Evaluation We evaluate our method using full-length F-1 ROUGE scores (Lin, 2004) and re-used the…”
Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation (2020, methodology): “The datasets were chosen as they all differ in their abstractiveness, output length (ranging from one sentence in XSum to on average four sentences in BigPatent), and cover multiple domains from news (CNNDM and XSum) to social media (Reddit) to patent documents (BigPatent) to show the generalizabili”
CTRLsum: Towards Generic Controllable Text Summarization (2020, methodology): “We use pretrained BART (Lewis et al., 2020) as the underlying architecture and perform experiments on three datasets: CNN/Dailymail news articles (Hermann et al., 2015), arXiv scientific papers (Cohan et al., 2018), and BIGPATENT patent documents (Sharma et al., 2019).”
Overview and Insights from the Shared Tasks at Scholarly Document Processing 2020: CL-SciSumm, LaySumm and LongSumm (2020, methodology): “For CL-SciSumm-20 we encourage participants to use out-of-domain data (i.e., scientiﬁc document corpora from papers outside of the ACL anthology corpora; e.g., BIGPATENT (Sharma et al., 2019)) to bootstrap training using transfer learning.”
A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis (2023, methodology): “We use arXiv (Cohan et al., 2018), PubMed (Cohan et al., 2018), BigPatent (Sharma et al., 2019), Multi-News (Fabbri et al., 2019), MediaSum (Zhu et al., 2021), CNN / Daily Mail (Nallapati et al., 2016) dataset for the evaluation, measuring the performance with ROUGE-1/2/L metrics.”
Efficient Attentions for Long Document Summarization (2021, background): “, 2018) that consist of scientific publications; BILLSUM (Kornilova and Eidelman, 2019), a collection of congressional bills; and BIGPATENT (Sharma et al., 2019), a corpus of”
BookSum: A Collection of Datasets for Long-form Narrative Summarization (2021, methodology): “Salient Content Distribution To assess the difficulty of content selection in our datasets we measure the distribution of salient unigrams in the source texts (Sharma et al., 2019).”
SQuALITY: Building a Long-Document Summarization Dataset the Hard Way (2022, background): “BigPatent (Sharma et al., 2019) and GovRe-1139 port (Huang et al., 2021) use expert-written summaries that come with patent  lings and government reports, respectively.”
Submix: Practical Private Prediction for Large-Scale Language Models (2022, methodology): “We split BigPatent-G by patent and deﬁne each user to be a single patent.”
Nomic Embed: Training a Reproducible Long Context Text Embedder (2024, unknown): “8 (Sharma et al., 2019).”
SummScreen: A Dataset for Abstractive Screenplay Summarization (2021, background): “…arXiv; Cohan et al., 2018), multiple news articles (Multi-News; Fabbri et al., 2019), opinionated text (RottenTomatoes; Wang and Ling, 2016), patents (Sharma et al., 2019), TV show stories (TVRecap; Chen and Gimpel, 2021) and (extractive summarization of) chapters of novels (Ladhak et al., 2020).”
LongT5: Efficient Text-To-Text Transformer for Long Sequences (2021, methodology): “BigPatent (Sharma et al., 2019) U.S. patent documents, with the patent’s details used as input and the patent’s abstract as the target summary.”