Unsupervised Construction of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources

By William B. Dolan, Chris Quirk, Chris Brockett. 2004

907 citations recorded by the source corpus.

We investigate unsupervised techniques for acquiring monolingual sentence-level paraphrases from a corpus of temporally and topically clustered news articles collected from thousands of web-based news sources. Two techniques are employed: (1) simple string edit distance, and (2) a heuristic strategy that pairs initial (presumably summary) sentences from different news stories in the same cluster. We evaluate both datasets using a word alignment algorithm and a metric borrowed from machine translation. Results show that edit distance data is cleaner and more easily-aligned than the heuristic data, with an overall alignment error rate (AER) of 11.58% on a similarly-extracted test set. On test data extracted by the heuristic strategy, however, performance of the two training sets is similar, with AERs of 13.2% and 14.7% respectively. Analysis of 100 pairs of sentences from each set reveals that the edit distance data lacks many of the complex lexical and syntactic alternations that characterize monolingual paraphrase. The summary sentences, while less readily alignable, retain more of the non-trivial alternations that are of greatest interest learning paraphrase relationships.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

An efficient framework for learning sentence representations (2018, neutral): “…sentiment (MR) (Pang & Lee, 2005), product reviews (CR) (Hu & Liu, 2004), subjectivity classiﬁcation (SUBJ) (Pang & Lee, 2004), opinion polarity (MPQA) (Wiebe et al., 2005), question type classiﬁ-cation (TREC) (Voorhees & Buckland, 2003) and paraphrase identiﬁcation (MSRP) (Dolan et al., 2004).”
SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation (2017, background): “Semantic inference tasks related to STS include textual entailment (Bentivogli et al., 2016; Bowman et al., 2015; Dagan et al., 2010), semantic relatedness (Bentivogli et al., 2016) and paraphrase detection (Xu et al., 2015; Ganitkevitch et al., 2013; Dolan et al., 2004).”
Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning (2017, methodology): “We evaluate the quality of the encoder learned by our system, which we call DiscSent, by using the sentence representations it produces in a variety of sentence classiﬁ-Model et al., 2004), subjectivity evaluation (SUBJ; Pang and Lee, 2004) and question classiﬁcation (TREC; Voorhees, 2001).”
Learning Distributed Representations of Sentences from Unlabelled Data (2016, methodology): “Representations are applied to 6 sentence classi-ﬁcation tasks: paraphrase identiﬁcation (MSRP) (Dolan et al., 2004), movie review sentiment (MR) (Pang and Lee, 2005), product reviews (CR) (Hu and Liu, 2004), subjectivity classiﬁcation (SUBJ) (Pang and Lee, 2004), opinion polarity (MPQA) (Wiebe et…”
Skip-Thought Vectors (2015, methodology): “The next task we consider is paraphrase detection on the Microsoft Research Paraphrase Corpus [30].”
SentEval: An Evaluation Toolkit for Universal Sentence Representations (2018, methodology): “eports both the average and the weighted average (by number of samples in each subtask) of the Pearson and Spearman correlations. Paraphrase detection The Microsoft Research Paraphrase Corpus (MRPC) (Dolan et al., 2004) is composed of pairs of sentences which have been extracted from news sources on”
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (2019, methodology): “• MRPC : Microsoft Research Paraphrase Corpus from parallel news sources (Dolan et al., 2004).”
Learning to Generate Reviews and Discovering Sentiment (2017, methodology): “The package received was blank and has no barcode.”
Semantic Parsing via Paraphrasing (2014, background): “In contrast, traditional para-phrase detection (Dolan et al., 2004) and Recognizing Textual Entailment (RTE) tasks (Dagan et al., 2013) consider examples consisting of only a single pair of candidate paraphrases.”
ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs (2015, methodology): “We use the Microsoft Research Paraphrase (MSRP) corpus (Dolan et al., 2004).”
Convolutional Neural Network for Paraphrase Identification (2015, methodology): “In experiments, CNN-LM is trained on unlabeled MSRP data and an additional 100,000 sentences from English Gigaword (Graff et al., 2003).”
Overcoming the Lack of Parallel Data in Sentence Compression (2013, neutral): “Similarly to previous work (Dolan et al., 2004; Wubben et al., 2009; Bejan & Harabagiu, 2010, inter alia), the Google News service 3 was used to identify news.”
Revisiting Recurrent Networks for Paraphrastic Sentence Embeddings (2017, neutral): “…will explore additional data sources, including from aligning different translations of novels (Barzilay and McKeown, 2001), aligning new articles of the same topic (Dolan et al., 2004), or even possibly using machine translation systems to translate bilingual text into paraphrastic sentence pairs.”
Discriminative Improvements to Distributional Sentence Similarity (2013, methodology): “On a standard paraphrase identiﬁcation task (Dolan et al., 2004), this method improves on both traditional TF-IDF and Weighted Textual Matrix Factorization (WTMF; Guo and Diab, 2012).”
Multi-Perspective Sentence Similarity Modeling with Convolutional Neural Networks (2015, methodology): “We demonstrate state-of-the-art performance on two SemEval semantic relatedness tasks (Agirre et al., 2012; Marelli et al., 2014), and highly competitive performance on the Microsoft Research paraphrase (MSRP) identiﬁcation task (Dolan et al., 2004).”
Unsupervised Learning of Sentence Embeddings Using Compositional n-Gram Features (2017, methodology): “For the MSRP dataset, containing pairs of sentences ( S 1 , S 2 ) with associated paraphrase label, we generate feature vectors by concatenating their Sent2Vec representations The predeﬁned training split is used to tune the L2 penalty parameter using cross-validation and the accuracy and F1 scores ”
ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations (2017, background): “ from their results. Paraphrase generation and discovery. Many researchers have developed methods for generating or ﬁnding naturally-occurring sentential paraphrase pairs (Barzilay and McKeown, 2001; Dolan et al., 2004; Dolan and Brockett, 2005; Quirk et al., 2004; Coster and Kauchak, 2011; Xu et al”
Learning Generic Sentence Representations Using Convolutional Neural Networks (2016, methodology): “Paraphrase detection Now we consider paraphrase detection on the MSRP dataset (Dolan et al., 2004).”
From Paraphrase Database to Compositional Paraphrase Model and Back (2015, background): “, 2014b), or entire sentences, such as the Microsoft Research Paraphrase Corpus (Dolan et al., 2004; Quirk et al., 2004).”