Adaptation Data Selection using Neural Language Models: Experiments in Machine Translation

By Kevin Duh, Graham Neubig, Katsuhito Sudoh, Hajime Tsukada. 2013

119 citations recorded by the source corpus.

Subfields: Large Language Models, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Large Language Models

In Natural Language Processing

Citations in the corpus

Simple, Scalable Adaptation for Neural Machine Translation (2019, background): “, 2018), performance on out-of-domain data or low resource languages, still remains pretty poor (Duh et al., 2013; Koehn and Knowles, 2017; Farajian et al., 2017; Dong et al., 2015; Zoph et al., 2016).”
Parallel Corpus Filtering via Pre-trained Language Models (2020, methodology): “Duh et al. (2013) use neural language models for data selection.”
Zipporah: a Fast and Scalable Data Cleaning System for Noisy Web-Crawled Parallel Corpora (2017, methodology): “2013) used a neural network based language model trained on a small in-domain corpus to select from a larger data pool.”
Learning to select data for transfer learning with Bayesian Optimization (2017, background): “Typically language models are leveraged via perplexity or cross-entropy scoring to select target data (Moore and Lewis, 2010; Axelrod et al., 2011; Duh et al., 2013; Mirkin and Besacier, 2014).”
Dynamic Data Selection for Neural Machine Translation (2017, positive): “Following related work by Moore and Lewis (2010), we restrict the vocabulary of the LMs to the words occurring at least twice in the in-domain corpus.”
Alibaba’s Neural Machine Translation Systems for WMT18 (2018, methodology): “N-grams: In order to acquire high-quality indomain data, we exploit the algorithm detailed in Duh et al. (2013); Axelrod et al. (2011), which aims at selecting sentence pairs from large outdomain corpus that are similar to the target domain.”
Accelerating Text Communication via Abbreviated Sentence Input (2021, methodology): “Duh et al. (2013) employed the data selection method of Axelrod et al. (2011), which builds upon Moore and Lewis (2010)’s approach.”
Bilingually-constrained Phrase Embeddings for Machine Translation (2014, background): “…et al., 2013), translation conﬁ-dence estimation (Mikolov et al., 2010; Liu et al., 2013; Zou et al., 2013), phrase reordering prediction (Li et al., 2013), translation modelling (Auli et al., 2013; Kalchbrenner and Blunsom, 2013) and language modelling (Duh et al., 2013; Vaswani et al., 2013).”
The JHU Parallel Corpus Filtering Systems for WMT 2018 (2018, methodology): “Axelrod et al. (2015) utilized part-of-speech tags and used a class-based n-gram language model for selecting in-domain data and Duh et al. (2013) used a neural network based language model trained on a small in-domain corpus to select from a larger mixed-domain data pool.”
Unsupervised Domain Clusters in Pretrained Language Models (2020, methodology): “Duh et al. (2013) proposed to replace n-gram models with RNN-based LMs with notable improvements.”
Class-based N-gram language difference models for data selection (2015, methodology): “by [3] using IBM model socres, edit distance [4], neural language models [5].”
Reinforcement Learning based Curriculum Optimization for Neural Machine Translation (2019, background): “Many approaches have been proposed to cope with heterogeneity, such as filtering (Duh et al., 2013) or down-weighting (Wang et al.”
A Survey of Domain Adaptation for Neural Machine Translation (2018, methodology): “It has been known that both vanilla SMT and NMT perform poorly for domain specific translation in low resource scenarios (Duh et al., 2013; Sennrich et al., 2013; Zoph et al., 2016; Koehn and Knowles, 2017).”
Incremental Adaptation Strategies for Neural Network Language Models (2014, background): “(selection of sentences in out-of-domain corpora based on similarity between sentences) and (Duh et al., 2013) (training of three models: n-gram, RNN and interpolated LM on two SMT systems: in-domain data only and all-domain).”
A Survey of Multilingual Neural Machine Translation (2019, background): “Both, vanilla SMT and NMT perform poorly for domain speciﬁc translation in low-resource scenarios (Duh et al., 2013; Koehn and Knowles, 2017).”
AutoSeM: Automatic Task Selection and Mixing in Multi-Task Learning (2019, methodology): “, 2017), language models (Moore and Lewis, 2010; Duh et al., 2013), dependency parsing (Søgaard, 2011), etc. In particular, Ruder and Plank (2017) used Bayesian optimization to select relevant training instances for transfer learning, and Tsvetkov et al.”
Domain Adaptation for Medical Text Translation using Web Resources (2014, background): “Perplexity-based data selection has shown to be a powerful mean on SMT domain adaptation (Wang et al., 2013; Wang et al., 2014; Toral, 2013; Rubino et al., 2013; Duh et al., 2013).”
Combining Domain Adaptation Approaches for Medical Text Translation (2014, methodology): “Therefore, we not 2: Statistics summary of corpora after pre-processing. only use the in-domain training data, but also the selected pseudo in-domain data 8 from general-domain corpus to enhance the LMs (Toral, 2013; Rubino et al., 2013; Duh et al., 2013).”