Retrieve, Rerank and Rewrite: Soft Template Based Neural Summarization

By Ziqiang Cao, Wenjie Li, Sujian Li, Furu Wei. 2018

206 citations recorded by the source corpus.

Most previous seq2seq summarization systems purely depend on the source text to generate summaries, which tends to work unstably. Inspired by the traditional template-based summarization approaches, this paper proposes to use existing summaries as soft templates to guide the seq2seq model. To this end, we use a popular IR platform to Retrieve proper summaries as candidate templates. Then, we extend the seq2seq framework to jointly conduct template Reranking and template-aware summary generation (Rewriting). Experiments show that, in terms of informativeness, our model significantly outperforms the state-of-the-art methods, and even soft templates themselves demonstrate high competitiveness. In addition, the import of high-quality external summaries improves the stability and readability of generated summaries.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

Citations in the corpus

Retrieve and Refine: Improved Sequence Generation Models For Dialogue (2018, background): “…(Junczys-Dowmunt and Grundkiewicz, 2017; Niehues et al., 2016; Novak et al., 2016; Xia et al., 2017; Grangier and Auli, 2017) or combining with retrieval (Gu et al., 2017; Cao et al., 2018), as well as for sentence correction or refinement without context (Guu et al., 2017; Schmaltz et al., 2017).”
Response Generation by Context-aware Prototype Editing (2018, methodology): “The other approach follows a retrieval-then-edit paradigm, that uses a Seq2Seq model to edit a prototype retrieved from a corpus (Guu et al. 2017; Li et al. 2018; Cao et al. 2018).”
Template Guided Text Generation for Task Oriented Dialogue (2020, methodology): “Our template based approach bears similarities to sentence fusion (Barzilay and McKeown, 2005), and prototype based text editing (Hossain et al., 2020; Cao et al., 2018; Guu et al., 2018; Wu et al., 2019).”
Automatic Document Sketching: Generating Drafts from Analogous Texts (2021, methodology): “Most use a retrieval-based method to extract similar references from the training corpus as prototypes (Cao et al., 2018; Yang et al., 2019; Wang et al., 2019; Gao et al., 2019; Peng et al., 2019), and learn to separate salient information and latent template structure.”
Distilling Knowledge Learned in BERT for Text Generation (2019, methodology): “Re 3 Sum (Cao et al., 2018a) carefully designed a retrieve-and-rerank pipeline with human-written soft templates.”
GSum: A General Framework for Guided Neural Abstractive Summarization (2020, background): “For example, Kikuchi et al. (2016) specify the length of abstractive summaries, Li et al. (2018) provide models with keywords to prevent the model from missing key information, and Cao et al. (2018) propose models that retrieve and reference relevant summaries from the training set.”
Controllable Paraphrase Generation with a Syntactic Exemplar (2019, background): “Another related line of work builds generation upon sentential exemplars (Guu et al., 2018; Weston et al., 2018; Pandey et al., 2018; Cao et al., 2018; Peng et al., 2019) in order to improve the quality of the generation itself, rather than to allow for control over syntactic structures.”
Case-based Reasoning for Natural Language Queries over Knowledge Bases (2021, background): “Another recent line of work use training examples at test time to improve language generation (Weston et al., 2018; Pandey et al., 2018; Cao et al., 2018; Peng et al., 2019).”
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training (2020, methodology): “…al., 2019), we compare our ProphetNet against following baselines: OpenNMT (Klein et al., 2017) which implements the standard Seq2Seq model with attention mechanism; Re2Sum (Cao et al., 2018) which employs an extended Seq2Seq model to generate summaries based on the retrieved candidate summaries.”
Syntax-Guided Controlled Generation of Paraphrases (2020, background): “summarization (Cao et al., 2018), and adversarial”
Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning (2021, unknown): “For the re-ranking models, fuzzy match score (Gu et al., 2018; Zhang et al., 2018; Hayati et al., 2018; Xu et al., 2020), attention mechanisms (Cao et al., 2018; Cai et al., 2019), and dot products between sentence representations (Lewis et al., 2020; Xu et al., 2020) are usual scoring functions to…”
A Survey of Knowledge-enhanced Text Generation (2020, methodology): “(i.e., answer) Creative writing Document - - [38] - [56, 70] - - - (e.g., story, paper) (e.g., paper title) [75, 149] Summarization Document [105, 146] [80, 84] [2, 104] [166] [65, 183] [72, 87] [84] [18, 47, 145] (e.g., an article) [43, 149] [81] Machine Sentence [152, 174] - [31, 124] - [103, 177]”
Multi-stage Pretraining for Abstractive Summarization (2019, unknown): “63 ReSum (Cao et al., 2018) 37.”
Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation (2018, methodology): “ection, relation generation, and content ordering. The task differs from image captioning [44, 24] and sentence generation [14, 6] where usually single or few sentences are required, or summarization [2, 45] where summaries tend to be more diverse without clear template sentences. State-of-the-art m”
Controlling Text Edition by Changing Answers of Specific Questions (2021, background): “Examples are the generation of summaries by template-based rewriting (Cao et al., 2018; Hashimoto et al., 2018) and the generation of text or a response by editing a prototype sentence (Guu et al., 2018; Pandey et al., 2018; Wu et al., 2019).”
Mimic and Rephrase: Reflective Listening in Open-Ended Dialogue (2019, background): “Such models have also been shown to be effective at semantic parsing (Jia and Liang, 2016), summarization (See et al., 2017; Cao et al., 2018), and task oriented”