Extracting Parallel Sentences from Comparable Corpora using Document Level Alignment

By Jason R. Smith, Chris Quirk, Kristina Toutanova. 2010

251 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Beyond English-Centric Multilingual Machine Translation (2020, background): “• FLORES — FLORES2 (Guzmán et al., 2019) pairs two low resource languages, Sinhala and Nepali, with English in the Wikipedia domain.”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, background): “ta, or text in different languages that are topically similar (i.e., “talk about the same thing”) but are not necessarily translations of each other [Resnik and Smith, 2003, Munteanu and Marcu, 2005, Smith et al., 2010]. Techniques that can take advantage of comparable corpora expand the scope and v”
WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia (2019, methodology): “In another work, the mining approach of Munteanu and Marcu (2005) was applied to extract large corpora from Wikipedia in sixteen languages (Smith et al., 2010).”
CCMatrix: Mining Billions of High-Quality Parallel Sentences on the Web (2019, methodology): “In another work, the mining approach of Munteanu and Marcu (2005) was applied to extract large corpora from Wikipedia in sixteen lan-2 guages (Smith et al., 2010).”
Neural CRF Model for Sentence Alignment in Text Simplification (2020, neutral): “Another related line of research (Smith et al., 2010; Tuﬁs , et al., 2013; Tsai and Roth, 2016; Gottschalk and Demidova, 2017; Aghaebrahimian, 2018; Thompson and Koehn, 2019) aligns parallel sentences in bilingual corpora for machine translation.”
Why Not Grab a Free Lunch? Mining Large Corpora for Parallel Sentences to Improve Translation Modeling (2012, methodology): “(Smith et al., 2010), we also report recall when precision was 80% and 90% in Table 1, and these values match the results of Smith et al.”
Crisis MT: Developing A Cookbook for MT in Crisis Situations (2011, background): “…has a long history, including mining comparable sources for named entities (Udupa et al., 2009; Irvine et al., 2010; Hewavitha-rana and Vogel, 2008; Hewavitharana and Vogel, 2011), mining Wikipedia for parallel content, including sentences (Smith et al., 2010), and many more too numerous to list.”
Wikipedia as an SMT Training Corpus (2013, background): “As mentioned in Section 2, Smith et al. (2010) mined for parallel sentences from Wikipedia producing parallel corpora of sizes similar to ours Furthermore, they have made their Wikipedia test set available for Spanish-English and German-English (500 sentence pairs per language pair).”
UM-pAligner: Neural Network-Based Parallel Sentence Identification Model (2018, methodology): “Various parallel sentence alignment models and strategies have also been applied to induce parallel sentences from the Wikipedia (Adafre and de Rijke, 2006; Yasuda and Sumita, 2008; Smith et al., 2010; Barr ´ on-Cede ˜ no et al., 2015).”
Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation (2021, background): “For instance, in Wikipedia, documents aligned across languages might contain parallel segments that share important content, yet they are not perfect translations of each other, yielding ﬁne-grained semantic divergences (Smith et al., 2010).”
Dirt Cheap Web-Scale Parallel Text from the Common Crawl (2013, methodology): “Many websites are available in multiple languages, and unlike other potential sources— such as multilingual news feeds (Munteanu and Marcu, 2005) or Wikipedia (Smith et al., 2010)— it is common to find document pairs that are direct translations of one another.”
Unsupervised Parallel Sentence Extraction with Parallel Segment Detection Helps Machine Translation (2019, methodology): “Similarly, parallel data was used to train IBM Model 1 and a maximum entropy classifier (Smith et al., 2010).”
Machine Translation With Weakly Paired Bilingual Documents (2018, background): “Smith et al. (2010) and Munteanu & Marcu (2005) develop a ranking model/binary classifier to learn how likely a sentence in target language is a translation of the source language using parallel corpora.”
Set-Theoretic Alignment for Comparable Corpora (2016, methodology): “The model we refer to as CRF (Smith et al., 2010) is a ﬁrst order linear chain Conditional Random Field (Lafferty et al., 2001), where for each source sentence a hidden variable indicates the corresponding target sentence to which it is aligned, or null if there is no such target sentence.”
IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation (2019, methodology): “It was previously used for unsupervised Statistical Machine Translation (SMT) (Fung and Yee, 1998; Munteanu et al., 2004; Smith et al., 2010) and Neural Machine Translation (NMT) (Conneau et al., 2017b; Lam-ple et al., 2017; Artetxe et al., 2017), where it iteratively takes the pseudo pairs to…”
Extracting Parallel Sentences from Comparable Corpora with STACC Variants (2018, background): “…2004), reference metrics over statistical machine translations (Abdul-Rauf and Schwenk, 2009; Sarikaya et al., 2009), feature-based approaches (Stef˘anescu et al., 2012; Smith et al., 2010) or deep learning with bidirectional recurrent neural networks (Gr ´ egoire and Langlais, 2017), among others.”
Crowdsourcing Translation: Professional Quality from Non-Professionals (2011, methodology): “These include harvesting the web for translations or comparable corpora (Resnik and Smith, 2003; Munteanu and Marcu, 2005; Smith et al., 2010; Uszkoreit et al., 2010), improving SMT models so that they are better suited to the low resource setting (Al-Onaizan et al., 2002; Probst et al., 2002; Oard…”