News from OPUS — A collection of multilingual parallel corpora with tools and interfaces

By Jörg Tiedemann. 2009

648 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Findings of the 2016 Conference on Machine Translation (2016, background): “We added a few new corpora: • Romanian Europarl (Koehn, 2002) • SETIMES2 from OPUS for Romanian– English and Turkish–English (Tiedemann, 2009) • Monolingual data sets from CommonCrawl (Buck et al., 2014) Some statistics about the training materials are given in Figure 1.”
Findings of the 2014 Workshop on Statistical Machine Translation (2014, methodology): “The medical-domain parallel data includes the following corpora (see Table 22 for statistics): The EMEA corpus (Tiedemann, 2009) contains documents from the European Medicines Agency, automatically processed and aligned on sentence level.”
A Diversity-Promoting Objective Function for Neural Conversation Models (2015, methodology): “Twitter conversations, we also used the OpenSub-titles (OSDb) dataset (Tiedemann, 2009), a large, noisy, open-domain dataset containing roughly 60M-70M scripted lines spoken by movie characters.”
Can Neural Machine Translation be Improved with User Feedback? (2018, background): “The out-of-domain data for training the baselines contains only publicly available parallel corpora, that is Europarl, TAUS, and OpenSubtitles released by the OPUS project (Tiedemann, 2009).”
A Persona-Based Neural Conversation Model (2016, methodology): “We nevertheless provide this result to give a sense that these B LEU scores of 5-6% are not unreasonable. models using a much larger OpenSubtitles (OSDb) dataset (Tiedemann, 2009), and then adapting the pre-trained model to the TV series dataset.”
Parallel Data, Tools and Interfaces in OPUS (2012, background): “Some of the resources have previously been presented (Tiedemann, 2009).”
A causal framework for explaining the predictions of black-box sequence-to-sequence models (2017, methodology): “To test this, we train a simple dialogue system on the OpenSubtitle corpus (Tiedemann, 2009), consisting of ∼14M two-step movie dialogues.”
A Simple, Fast Diverse Decoding Algorithm for Neural Generation (2016, methodology): “We used the OpenSubtitles (OSDb) dataset (Tiedemann, 2009), an open-domain movie script dataset containing roughly 60M-70M scripted lines spoken by movie characters.”
DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset (2017, methodology): “The most widely adopted dataset is OpenSubtitle (J¨org Tiedemann, 2009) which is used in Li et al. (2016, 2017).”
Regularized Training Objective for Continued Training for Domain Adaptation in Neural Machine Translation (2018, methodology): “We draw inspiration from a number of prior works including Yu et al. (2013), which introduces Kullback-Leibler (KL) divergence between the model being trained and an out-of-domain model as a regularization scheme for speaker adaptation.”
Non-Parametric Adaptation for Neural Machine Translation (2019, methodology): “• We evaluate our approach on a multi-domain English-French corpus constructed from narrow domain datasets like JRC-Acquis (Stein-berger et al., 2006; Tiedemann) and Open-Subtitles (Tiedemann, 2009) 1 , and the standard IWSLT and WMT bilingual corpora, as described in Sections 3 and 4.”
On the Impact of Various Types of Noise on Neural Machine Translation (2018, methodology): “8 The SHORT SEGMENTS are extracted from OPUS corpora (Tiedemann, 2009, 2012; Lison and Tiedemann, 2016): 9 EMEA (descriptions of medicines), 10 Tanzil (religious text), 11 Open Sub-titles 2016, 12 Acquis (legislative text), 13 GNOME (software localization ﬁles), 14 KDE (localization ﬁles), PHP…”
An Empirical Analysis of NMT-Derived Interlingual Embeddings and Their Use in Parallel Sentence Identification (2017, methodology): “The parallel corpus includes data from United Nations (Chen and Eisele, 2012), Common Crawl, Europarl (Koehn, 2005), EMEA (Tiedemann, 2009) and Scielo 7 .”
Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints (2018, methodology): “Due to the noisy nature of the OpenSubtitles conversations we do not use them for evaluation.”
Detecting egregious responses in neural sequence-to-sequence models (2018, methodology): “In addition to the Ubuntu and Switchboard data-sets, we also report experiments on the Open-Subtitles data-set 6 (Tiedemann, 2009).”
Character-Based Pivot Translation for Under-Resourced Languages and Domains (2012, methodology): “We chose to use data from a collection of translated subtitles compiled in the freely available OPUS corpus (Tiedemann, 2009b).”