Paraphrasing Revisited with Neural Machine Translation

By Mirella Lapata, Rico Sennrich, Jonathan Mallinson. 2017

224 citations recorded by the source corpus.

Recognizing and generating paraphrases is an important component in many natural language processing applications. A well-established technique for automatically extracting paraphrases leverages bilingual corpora to find meaning-equivalent phrases in a single language by “pivoting” over a shared translation in another language. In this paper we revisit bilingual pivoting in the context of neural machine translation and present a paraphrasing model based purely on neural networks. Our model represents paraphrases in a continuous space, estimates the degree of semantic relatedness between text segments of arbitrary length, and generates candidate paraphrases for any source input. Experimental results across tasks and datasets show that neural paraphrases outperform those obtained with conventional phrase-based pivoting approaches.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension (2018, methodology): “While the concept of backtranslation has been introduced before, it is often used to improve either the same translation task Sennrich et al. (2016) or instrinsic paraphrase evaluations Wieting et al. (2017); Mallinson et al. (2017).”
How Can We Know What Language Models Know? (2019, methodology): “method of using back-translation (Sennrich et al., 2016; Mallinson et al., 2017) to first translate the initial prompt into B candidates in another language, each of which is then back-translated into B candidates in the original language.”
Semantically Equivalent Adversarial Rules for Debugging NLP models (2018, methodology): “We turn instead to paraphrasing based on neural machine translation (Lapata et al., 2017), where P (x′|x) (the probability of a paraphrase x′ given original sentence x) is proportional to translating x into multiple pivot languages”
Adversarial Example Generation with Syntactically Controlled Paraphrase Networks (2018, methodology): “Mallinson et al. (2017) show that neural machine translation (NMT) systems outperform phrase-based MT on several paraphrase evaluation metrics.”
Ask the Right Questions: Active Question Reformulation with Reinforcement Learning (2017, methodology): “ases as a latent representation. Bilingual corpora and MT have been used to generate paraphrases by pivoting through a second language. Recent work uses neural translation models and multiple pivots (Mallinson et al., 2017). In contrast, our approach does not use pivoting and is, to our knowledge, t”
Style Transfer Through Back-Translation (2018, methodology): “More recently, Mallinson et al. (2017) and Wieting et al. (2017) showed how neural back-translation can be used to generate paraphrases.”
ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations (2017, methodology): “nch system. Hill et al. (2016) evaluated the encoders of English-to-X NMT systems as sentence representations, ﬁnding them to perform poorly compared to several other methods based on unlabeled data. Mallinson et al. (2017) adapted trained NMT models to produce sentence similarity scores in semantic”
Learning Paraphrastic Sentence Embeddings from Back-Translated Bitext (2017, methodology): “Mallinson et al. (2017) adapted trained NMT models to produce sentence similarity scores in semantic evaluations.”
Text Repair Model for Neural Machine Translation (2019, methodology): “Mallinson et al. (2017) presented PARANET, a neural paraphrasing model based on round-trip translations with NMT.”
AutoQA: From Databases to Q&A Semantic Parsers with Only Synthetic Training Data (2020, methodology): “We cannot 2Most large-scale paraphrasing datasets are built using bilingual text (Ganitkevitch et al., 2013) and machine translation (Mallinson et al., 2017) or obtained with noisy heuristics (Prakash et al., 2016).”
Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations (2017, methodology): “Mallinson et al. (2017) adapted trained NMT models to produce sentence similarity scores in semantic evaluations.”
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization (2021, methodology): “To create multiple positive samples, in our pilot study, we experiment with paraphrasing with synonym substitution (Ren et al., 2019), randomly replacing words based on the prediction of masked language models (Kobayashi, 2018), and back-translation (Mallinson et al., 2017).”
Controllable Paraphrase Generation with a Syntactic Exemplar (2019, background): “We focus primarily on the task of paraphrase generation, which has received significant recent attention (Quirk et al., 2004; Prakash et al., 2016; Mallinson et al., 2017; Dong et al., 2017; Ma et al., 2018; Li et al., 2018).”
User Utterance Acquisition for Training Task-Oriented Bots: A Review of Challenges, Techniques and Opportunities (2020, methodology): “In the absence of monolingual paraphrase datasets, the typical approach is to use language pivoting.(20) In language pivoting, a given utterance is translated into another language, and then translated back to the source language to obtain a paraphrase.”
An approximate sampler for energy-based models with divergence diagnostics (2022, methodology): “In particular, we use a round-trip machinetranslation model, which is a well-known tool in generating paraphrases (Bannard and Callison-Burch, 2005; Mallinson et al., 2017).”
Learning to Paraphrase for Question Answering (2017, unknown): “Instead, we translate from multiple pivot sentences (Mallinson et al., 2016).”
CODIT: Code Editing With Tree-Based Neural Models (2018, background): “Similar ideas have been explored in NLP for paraphrasing [35].”
Split and Rephrase (2017, background): “Sentence paraphrasing aims to rewrite a sentence while preserving its meaning (Dras, 1999; Barzilay and McKeown, 2001; Bannard and Callison-Burch, 2005; Wubben et al., 2010; Mallinson et al., 2017).”
Semantic Equivalent Adversarial Data Augmentation for Visual Question Answering (2020, methodology): “character or word change, would easily destroy the grammar and semantics, rendering the possibility of attack failure. Adhere to the principle of not changing the semantics of input data, inspired by [15,28], we generate semantically equivalent adversarial questions by using a sequenceto-sequence pa”