Large Scale Parallel Document Mining for Machine Translation

By Jakob Uszkoreit, Jay M. Ponte, Ashok Popat, Moshe Dubiner. 2010

167 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges (2019, background): “Our problem signiﬁcantly extends those studied by these previous works: we study multilingual NMT on a massive scale, using an in-house corpus generated by crawling and extracting parallel sentences from the web (Uszkoreit et al., 2010).”
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding (2020, neutral): “The training corpus, mined from the web [47], contains parallel documents for 100 languages, to and from English, adding up to a total of 25 billion training examples.”
Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation (2019, methodology): “Data We train our multilingual NMT system on a massive scale, using an in-house corpus generated by crawling and extracting parallel sentences from the web (Uszkoreit et al., 2010).”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “at the machine translation problem, since the restriction for semantic equivalence is relaxed. Furthermore, researchers have developed techniques for mining comparable corpora automatically at scale [Uszkoreit et al., 2010, Ture and Lin, 2012]. These can be viewed as a cross-lingual text ranking pro”
Cross-lingual Retrieval for Iterative Self-Supervised Training (2020, background): “There is also a large area of research in using sentence-level representations to mine pseudo-parallel sentence pairs [6, 13, 19, 22, 48, 49, 52].”
Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings (2018, methodology): “Nevertheless, it has been argued that metadata information is not always available nor necessarily reliable (Uszkoreit et al., 2010), which has motivated alternative methods that rely on the textual content of the comparable documents instead.”
ParaCrawl: Web-Scale Acquisition of Parallel Corpora (2020, background): “While the idea of mining the web for parallel data has been already pursued in the 20th century (Resnik, 1999), the most serious efforts have been limited to large companies such as Google (Uszkoreit et al., 2010) and Microsoft (Rarrick et al., 2011), or targeted efforts on speciﬁc domains such as t”
Investigating Multilingual NMT Representations at Scale (2019, methodology): “We study multilingual NMT on a massive scale, using an in-house training corpus (Arivazhagan et al., 2019b) generated by crawling and extracting parallel sentences from the web (Uszkoreit et al., 2010).”
Effective Parallel Corpus Mining using Bilingual Sentence Embeddings (2018, methodology): “Then, we present results for document level matching using Uszkoreit et al. (2010)’s method as a strong base-line.”
Deep Learning for Text Style Transfer: A Survey (2020, methodology): “Such retrieval-based pseudoparallel data construction is also useful for machine translation (Munteanu and Marcu 2005; Uszkoreit et al. 2010; Marie and Fujita 2017; Grégoire and Langlais 2018; Ren et al. 2020).”
Dirt Cheap Web-Scale Parallel Text from the Common Crawl (2013, methodology): “Uszkoreit et al. (2010), for example, translated all non-English webpages into English using an existing translation system and used near-duplicate detection methods to ﬁnd candidate parallel document pairs.”
Building Machine Translation Systems for the Next Thousand Languages (2022, background): “These improvements have largely been driven by advances in machine learning and the availability of large-scale web-mined datasets (Resnik & Smith, 2003; Uszkoreit et al., 2010; Esplà-Gomis, 2009; Esplà et al., 2019; Bañón et al., 2020; Schwenk et al., 2021).”
Ernest: Efficient Performance Prediction for Large-Scale Advanced Analytics (2016, background): “…decade we have seen a rapid growth of large-scale advanced analytics that implement complex algorithms in areas like distributed natural language processing [24, 74], deep learning for image recognition [34], genome analysis [72, 61], astronomy [17] and particle accelerator data processing [19].”
Parallel Sentence Mining by Constrained Decoding (2020, background): “A typical workﬂow reduces the search complexity in a coarse-to-ﬁne manner by aligning documents then aligning sentences within documents (Uszkoreit et al., 2010).”
Why Not Grab a Free Lunch? Mining Large Corpora for Parallel Sentences to Improve Translation Modeling (2012, background): “A recent study from Google describes a general solution to our problem that scales to web collections (Uszkoreit et al., 2010).”
Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models (2020, methodology): “…multi-task optimization on a massive scale, we use an in-house training corpus 1 (Arivazhagan et al., 2019) generated by crawling and extracting parallel sentences from the web (Uszkoreit et al., 2010), which contains more than 25 billion sentence pairs for 102 languages to and from English.”
Findings of the WMT 2016 Bilingual Document Alignment Shared Task (2016, background): “Common choices include edit-distance between linearized documents (Resnik and Smith, 2003), cosine distance of idfweighted bigram vectors (Uszkoreit et al., 2010), and probability of a probabilistic DOM-tree alignment model (Shi et al.”