Curriculum Learning for Domain Adaptation in Neural Machine Translation

By Xuan Zhang, Pamela Shapiro, Manish Kumar, Paul McNamee, Marine Carpuat, Kevin Duh. 2019

131 citations recorded by the source corpus.

We introduce a curriculum learning approach to adapt generic neural machine translation models to a specific domain. Samples are grouped by their similarities to the domain of interest and each group is fed to the training algorithm with a particular schedule. This approach is simple to implement on top of any neural framework or architecture, and consistently outperforms both unadapted and adapted baselines in experiments with two distinct domains and two language pairs.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Survey on Curriculum Learning (2021, methodology): “This measurement of domain score is leveraged in [118], [139] as Transfer Teacher according to the speciﬁc scenarios (e.g., in-domain data can be seen as easier for domain adaption).”
Unsupervised Domain Clusters in Pretrained Language Models (2020, methodology): “, 2020), using such selection methods with domain-curriculum training (Zhang et al., 2019; Wang et al., 2019b), applying them on noisy, web-crawled data (Junczys-Dowmunt, 2018) or for additional tasks (Gururangan et al.”
Human-in-the-loop machine learning: a state of the art (2022, methodology): “Nevertheless, some hybrid CL methods are designed which adopt different CL methods making them complement to each other (Jiang et al. 2015; Zhang et al. 2019a).”
Reinforcement Learning based Curriculum Optimization for Neural Machine Translation (2019, background): “Several recent papers (Kocmi and Bojar, 2017; Zhang et al., 2019; Platanios et al., 2019) explore similar ideas for NMT, and verify that this strategy can reduce training time and improve quality.”
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks (2020, background): “Data selection for transfer learning Selecting data for transfer learning has been explored in NLP (Moore and Lewis, 2010; Ruder and Plank, 2017; Zhang et al., 2019, among others).”
Curriculum Learning: A Survey (2021, methodology): “NAACL-HLT 2019 [136] NLP DA, machine translation CL distance from source domain batches data”
Self-Induced Curriculum Learning in Self-Supervised Neural Machine Translation (2020, methodology): “Zhang et al. (2019) adapt an NMT model to a domain by introducing increasingly domain-distant ( difﬁcult ) samples.”
Data Kernel Perspective Space Performance Guarantees for Synthetic Data from Transformer Models (2026, unknown): “Domain sensitivity is a commonly observed phenomenon in testing and evaluation of transformer-based language technology applications, including those built from large language models (LLMs) [25], [26].”