Text Generation with Exemplar-based Adaptive Decoding

By Hao Peng, Ankur P. Parikh, Manaal Faruqui, Bhuwan Dhingra, Dipanjan Das. 2019

59 citations recorded by the source corpus.

We propose a novel conditioned text generation model. It draws inspiration from traditional template-based text generation techniques, where the source provides the content (i.e., what to say), and the template influences how to say it. Building on the successful encoder-decoder paradigm, it first encodes the content representation from the given input text; to produce the output, it retrieves exemplar text from the training data as “soft templates,” which are then used to construct an exemplar-specific decoder. We evaluate the proposed model on abstractive text summarization and data-to-text generation. Empirical results show that this model achieves strong performance and outperforms comparable baselines.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Tailor: Generating and Perturbing Text with Semantic Controls (2021, methodology): “Controllable text generation has been widely used to inﬂuence various properties of generated text for text summarization (Peng et al., 2019), data augmentation (Lee et al., 2021), style transfer (Reid and Zhong, 2021; Madaan et al., 2020a), adversarial example generation (Iyyer et al., 2018), etc.”
What Makes Good In-Context Examples for GPT-3? (2021, background): “…dialogue generation (Yan et al., 2016; Cai et al., 2018; Song et al., 2016; Pandey et al., 2018; We-ston et al., 2018; Wu et al., 2019), text summarization (Cao et al., 2017; Peng et al., 2019), data-to-text generation (Peng et al., 2019), and text-to-code generation (Hashimoto et al., 2018).”
ToTTo: A Controlled Table-To-Text Generation Dataset (2020, methodology): “The technique of editing exemplar sentences has been used in semiparametric generation models (Guu et al., 2018; Pandey et al., 2018; Peng et al., 2019) and crowd-sourcing small, iterative changes to text has been shown to lead to higher-quality data and a more robust annotation process (Little et…”
Case-based Reasoning for Natural Language Queries over Knowledge Bases (2021, background): “Another recent line of work use training examples at test time to improve language generation (Weston et al., 2018; Pandey et al., 2018; Cao et al., 2018; Peng et al., 2019).”
Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation (2019, methodology): “Other approaches have proposed generating more accurate text using semiparametric approaches (Guu et al., 2018; Pandey et al., 2018; Peng et al., 2019), reinforcement learning-based rewards (Paulus et al., 2018; Pasunuru & Bansal, 2018), semi-Markov models to learn neural templates (Wiseman et al.,…”
Controllable Semantic Parsing via Retrieval Augmentation (2021, methodology): “And some use the retrievals to explicitly manipulate the token scores at each decoding step (Zhang et al., 2018; Hayati et al., 2018; Peng et al., 2019; Khandelwal et al., 2020, 2021).”
Compound Probabilistic Context-Free Grammars for Grammar Induction (2019, methodology): “It is possible that the model is utilizing the subtrees to capture broad template-like structures and then using z to ﬁll them in, similar to recent works that also train models to separate “what to say” from “how to say it” (Wiseman et al., 2018; Peng et al., 2019; Chen et al., 2019a,b).”
Controllable Paraphrase Generation with a Syntactic Exemplar (2019, background): “Another related line of work builds generation upon sentential exemplars (Guu et al., 2018; Weston et al., 2018; Pandey et al., 2018; Cao et al., 2018; Peng et al., 2019) in order to improve the quality of the generation itself, rather than to allow for control over syntactic structures.”
Reformulating Unsupervised Style Transfer as Paraphrase Generation (2020, methodology): “Syntax can also be controlled using an exemplar sentence (Chen et al., 2019; Guu et al., 2018; Peng et al., 2019).”
Automatic Document Sketching: Generating Drafts from Analogous Texts (2021, methodology): “Most use a retrieval-based method to extract similar references from the training corpus as prototypes (Cao et al., 2018; Yang et al., 2019; Wang et al., 2019; Gao et al., 2019; Peng et al., 2019), and learn to separate salient information and latent template structure.”
Syntax-Guided Controlled Generation of Paraphrases (2020, background): “…information has shown promising results in neural machine translation (Stahlberg et al., 2016; Aharoni and Goldberg, 2017; Yang et al., 2019), data-to-text generation (Peng et al., 2019), abstractive text-summarization (Cao et al., 2018) and adversarial text generation (Iyyer et al., 2018).”