Open Subtitles Paraphrase Corpus for Six Languages

By Mathias Creutz. 2018

80 citations recorded by the source corpus.

This paper accompanies the release of Opusparcus, a new paraphrase corpus for six European languages: German, English, Finnish, French, Russian, and Swedish. The corpus consists of paraphrases, that is, pairs of sentences in the same language that mean approximately the same thing. The paraphrases are extracted from the OpenSubtitles2016 corpus, which contains subtitles from movies and TV shows. The informal and colloquial genre that occurs in subtitles makes such data a very interesting language resource, for instance, from the perspective of computer assisted language learning. For each target language, the Opusparcus data have been partitioned into three types of data sets: training, development and test sets. The training sets are large, consisting of millions of sentence pairs, and have been compiled automatically, with the help of probabilistic ranking functions. The development and test sets consist of sentence pairs that have been checked manually; each set contains approximately 1000 sentence pairs that have been verified to be acceptable paraphrases by two annotators.

Subfields: Artificial Intelligence, Language and Linguistics, Natural Language Processing, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Language and Linguistics

In Natural Language Processing

In Machine Translation

Citations in the corpus

PAWS: Paraphrase Adversaries from Word Scrambling (2019, background): “, 2017), subtitles (Creutz, 2018), and back translation (Iyyer et al.”
PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification (2019, background): “Existing multilingual datasets for paraphrase iden-tiﬁcation, e.g. Multi30k (Elliott et al., 2016) and Opusparcus (Creutz, 2018), lack challenging examples like PAWS.”
PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction (2020, methodology): “Paraphrase corpus As additional training data, we use the corpus of automatically aligned para-phrases of TV subtitles (Creutz, 2018, Para.).”