Domain Adaptation via Pseudo In-Domain Data Selection

By Amittai Axelrod, Xiaodong He, Jianfeng Gao. 2011

579 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Neural Machine Translation by Jointly Learning to Align and Translate (2014, methodology): “(2014a), we reduce the size of the combined corpus to have 348M words using the data selection method by Axelrod et al. (2011).5 We do not use any monolingual data other than the mentioned parallel corpora, although it may be possible to use a much larger monolingual corpus to pretrain an encoder.”
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation (2014, neutral): “…in (Koehn et al., 2003) and (Marcu and Wong, 2002), the translation model log p(e | f) is factorized into the translation probabilities of matching phrases in the source and target sentences.2 These probabilities are once again considered additional features in the log-linear model (see Eq.”
Edinburgh’s Phrase-based Machine Translation Systems for WMT-14 (2014, methodology): “We also experimented with modified MooreLewis (Moore and Lewis, 2010; Axelrod et al., 2011) data selection, using the EMEA corpus as the in-domain corpus (for the language model required in MML) and selecting from all the out-ofdomain data.”
Finetuned Language Models Are Zero-Shot Learners (2021, background): “…on MTL for NLP. Differing from prior MTL work which focuses on performance improvements across training tasks (Raffel et al., 2020; Aghajanyan et al., 2021) or to new domains (Axelrod et al., 2011), our work is motivated by improving zero-shot generalization to tasks that were not seen in training.”
Tagged Back-Translation (2019, methodology): “(2017); Axelrod et al. (2011), or curriculum learning approaches like fine-tuning on in-domain data (Thompson et al.”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, methodology): “bove. Domain adaptation using data selection tricks to choose portions of a larger dataset that are closer to the target test data have been explored previously in the context of machine translation [Axelrod et al., 2011], but it was quite surprising that simple keyword-based ﬁltered performed so we”
Parallel Corpus Filtering via Pre-trained Language Models (2020, background): “Axelrod et al. (2011) and Axelrod et al. (2015) expand cross-entropy difference filtering to both sides of”
Filtering and Mining Parallel Data in a Joint Multilingual Space (2018, background): “The problem of how to select parts of bitexts has been addressed before, but mainly from the aspect of domain adaptation (Axelrod et al., 2011; San-tamar´ıa and Axelrod, 2017).”
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches (2014, methodology): “We use the bilingual, parallel corpus which is a set of 348M selected by the method in (Axelrod et al., 2011) from a combination of Europarl (61M words), news commentary (5.”
Achieving Human Parity on Automatic Chinese to English News Translation (2018, methodology): “Then we apply Cross-Entropy data selection [27] and [2] to order the sentences based on their relevance to the CWMT part of the WMT data.”
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets (2021, background): “, 2014), to neural approaches like (contrastive) language-model scoring (Axelrod et al., 2011; Moore and Lewis, 2010; Wang et al., 2018).”
Fast Domain Adaptation for Neural Machine Translation (2016, methodology): “In SMT, this has been done for language model training [Gao et al., 2002, Moore and Lewis, 2010] and translation model training [Matsoukas et al., 2009, Foster et al., 2010, Axelrod et al., 2011, Mansour and Ney, 2014].”
Learning to select data for transfer learning with Bayesian Optimization (2017, methodology): “While this has been shown to work empirically (Moore and Lewis, 2010; Axelrod et al., 2011; Plank and van Noord, 2011; Van Asch and Daele-mans, 2010; Remus, 2012), using a pre-existing metric leaves us unable to adapt to the characteristics of our task T and target domain D T and fore-goes…”
Mind the Gap: Assessing Temporal Generalization in Neural Language Models (2021, background): “OoD generalization is well-studied in NLP (Blitzer et al., 2006; Daumé III, 2007; Axelrod et al., 2011), and has recently been addressed for neural LMs and transfer learning (Fried et al., 2019; Oren et al., 2019; Gururangan et al., 2020), where pretrained LMs lead to substantial improvements and…”
Dynamic Data Selection for Neural Machine Translation (2017, methodology): “In summary, our contributions are as follows: (i) We compare the effects of a commonly used data selection approach (Axelrod et al., 2011) on PBMT and NMT using four different test sets.”
Overcoming the Curse of Sentence Length for Neural Machine Translation using Automatic Segmentation (2014, methodology): “We use a bilingual, parallel corpus of 348M words selected by the method of (Axelrod et al., 2011) from a combination of Europarl (61M), news commentary (5.”
Findings of the WMT 2018 Shared Task on Parallel Corpus Filtering (2018, background): “There is a rich literature on data selection which aims at sub-sampling parallel data relevant for a task-specific machine translation system (Axelrod et al., 2011).”
On the Impact of Various Types of Noise on Neural Machine Translation (2018, background): “There is a rich literature on data selection which aims at sub-sampling parallel data relevant for a task-speciﬁc machine translation system (Axelrod et al., 2011). van der Wees et al. (2017) ﬁnd that the existing data selection methods developed for statistical machine translation are less…”