A Continuously Growing Dataset of Sentential Paraphrases

By Wuwei Lan, Siyu Qiu, Hua He, Wei Xu. 2017

181 citations recorded by the source corpus.

A major challenge in paraphrase research is the lack of parallel corpora. In this paper, we present a new method to collect large-scale sentential paraphrases from Twitter by linking tweets through shared URLs. The main advantage of our method is its simplicity, as it gets rid of the classifier or human in the loop needed to select data before annotation and subsequent application of paraphrase identification algorithms in the previous work. We present the largest human-labeled paraphrase corpus to date of 51,524 sentence pairs and the first cross-domain benchmarking for automatic paraphrase identification. In addition, we show that more than 30,000 new sentential paraphrases can be easily and continuously captured every month at ~70% precision, and demonstrate their utility for downstream NLP tasks through phrasal paraphrase extraction. We make our code and data freely available.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Adversarial NLI: A New Benchmark for Natural Language Understanding (2019, methodology): “Re-latedly, Lan et al. (2017) propose a method for continuously growing a dataset of paraphrases.”
PAWS: Paraphrase Adversaries from Word Scrambling (2019, background): “Existing data creation techniques have focused on collecting paraphrases, e.g. from co-captions for images (Lin et al., 2014), tweets with shared URLs (Lan et al., 2017), subtitles (Creutz, 2018), and back translation (Iyyer et al., 2018).”
ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations (2017, background): “, 2017), and using tweets with matching URLs (Lan et al., 2017).”
Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks (2019, methodology): “We use Tweets (Lan et al., 2017) data set and the Quora paraphrase data set (Wang et al., 2017).”
Submodular Optimization-based Diverse Paraphrasing and its Effectiveness in Data Augmentation (2019, methodology): “Recently sequence-tosequence based data-driven deep learning models have been proposed, which try to address the limitations of earlier traditional rule-based (McKeown, 1983) methods. Prakash et al. (2016) employ residual connections in LSTM to enhance the traditional sequence-to-sequence model.”
The BQ Corpus: A Large-scale Domain-specific Chinese Corpus For Sentence Semantic Equivalence Identification (2018, background): “…several corpora have been provided in recent years, including the Microsoft Research Paraphrase (MSRP) Corpus (Dolan et al., 2004; Dolan and Brockett, 2005), the Twitter Paraphrase Corpus (PIT-2015 corpus) (Xu et al., 2014, 2015), the Twitter URL corpus (Lan et al., 2017) and the Quora dataset 2.”
Transfer Fine-Tuning: A BERT Case Study (2019, background): “• Twitter URL corpus (Lan et al., 2017): This corpus was collected from Twitter by linking tweets through shared URLs.”
Reflective Decoding: Beyond Unidirectional Generation with Off-the-Shelf Language Models (2020, methodology): “We include here the full results for paraphrasing on the Twitter URL corpus (Lan et al., 2017), a set of paraphrase pairs created by linking tweets with matching shared URLs.”
LCQMC:A Large-scale Chinese Question Matching Corpus (2018, methodology): “The other paraphrase corpora mainly come from SemEval task, e.g. the semantic textual similarity corpus(SRA, (Dzikovska et al., 2013)), semantic textual similarity (Agirre et al., 2013), and Twitter paraphrase corpus (Xu et al., 2015; Lan et al., 2017).”
Paraphrase Generation: A Survey of the State of the Art (2021, background): “MSCOCO is more preferred for less noise compared with Twitter URL and ParaNMT.”
Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations (2017, background): “Many researchers have developed methods for generating or finding naturally-occurring sentential paraphrase pairs (Barzilay and McKeown, 2001; Dolan et al., 2004; Dolan and Brockett, 2005; Quirk et al., 2004; Coster and Kauchak, 2011; Xu et al., 2014, 2015; Lan et al., 2017).”
MTEB: Massive Text Embedding Benchmark (2022, unknown): “(d) Reproducibility: Through versioning at a dataset and software level, we aim to make it easy to repro- MTEB 8 Tasks 58 Datasets Massive Text Embedding Benchmark Classification AmazonCounterfactual Retrieval Pair Classification AmazonPolarity AmazonReviews Banking77 Emotion Imdb MassiveIntent Mass”
Calibration of Pre-trained Transformers (2020, background): “Our outof-domain setting is TwitterPPDB (TPPDB), which contains sentence pairs from Twitter where tweets are considered paraphrases if they have shared URLs (Lan et al., 2017).”
Simple and Effective Paraphrastic Similarity from Parallel Translations (2019, background): “However, such datasets are not produced naturally at scale and therefore must be created either through costly manual annotation or by leveraging natural annotation in speciﬁc domains, like Simple English Wikipedia (Coster and Kauchak, 2011) or Twitter (Lan et al., 2017).”
Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs (2026, unknown): “…Touche2020 Retrieval nDCG@10 TRECCOVID [57] Retrieval nDCG@10 Other (6) AskUbuntuDupQuestions Reranking MAP MindSmallReranking [74] Reranking MAP SICK-R [42] STS Spearman STSBenchmark [7] STS Spearman SprintDuplicateQuestions [59] Duplicate detection AP TwitterURLCorpus [34] Duplicate detection AP”