Adversarial Adaptation of Synthetic or Stale Data

By Young-Bum Kim, Karl Stratos, Dongchan Kim. 2017

50 citations recorded by the source corpus.

Two types of data shift common in practice are 1. transferring from synthetic data to live user data (a deployment shift), and 2. transferring from stale data to current data (a temporal shift). Both cause a distribution mismatch between training and evaluation, leading to a model that overfits the flawed training data and performs poorly on the test data. We propose a solution to this mismatch problem by framing it as domain adaptation, treating the flawed training dataset as a source domain and the evaluation dataset as a target domain. To this end, we use and build on several recent advances in neural domain adaptation such as adversarial training (Ganinet al., 2016) and domain separation network (Bousmalis et al., 2016), proposing a new effective adversarial training scheme. In both supervised and unsupervised adaptation scenarios, our approach yields clear improvement over strong baselines.

Subfields: Artificial Intelligence, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Towards Open Intent Discovery for Conversational Text (2019, background): “, to discover features and structures that are common across multiple domains [9, 25, 32].”
Well-Read Students Learn Better: The Impact of Student Initialization on Knowledge Distillation (2019, background): “In addition, for deployed systems, input data distribution shifts over time and existing unlabeled data becomes stale (Kim et al., 2017).”
ONENET: Joint domain, intent, slot prediction for spoken language understanding (2017, background): “A typical way to handle multiple domains for the spoken language understanding (SLU) task [1] is to perform domain prediction ﬁrst and then carry out intent prediction [2, 3, 4] and slot tagging [5, 6, 7, 8, 9, 10, 3, 4] (Figure 1).”
Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling (2019, methodology): “However, adversarial methods require balancing between at least two and as many as six different objectives (Kim et al., 2017), which can lead to instability (Arjovsky et al., 2017) unless the objectives are carefully balanced (Alam et al., 2018).”
Strong Baselines for Neural Semi-Supervised Learning under Domain Shift (2018, methodology): “Our MT-Tri model lends itself well to shared-private models such as those proposed recently (Liu et al., 2017; Kim et al., 2017), which extend upon (Ganin et al.”
Well-Read Students Learn Better: On the Importance of Pre-training Compact Models (2019, background): “nference and question answering, as well as domain-speciﬁc dialog understanding. In addition, for deployed systems, input data distribution shifts over time and existing unlabeled data becomes stale (Kim et al., 2017). Unlabeled language model data (D LM) is a collection of natural language texts th”
Neural Unsupervised Domain Adaptation in NLP—A Survey (2020, background): “As noted also by Han and Eisenstein (2019), a downside of adversarial methods is that they require careful balancing between objectives (Kim et al., 2017; Alam et al., 2018a) to avoid instability during learning (Arjovsky et al.”
KinGDOM: Knowledge-Guided DOMain Adaptation for Sentiment Analysis (2020, background): “Rectiﬁcations include shared-private encoders that model both domain-invariant and - speciﬁc features (Li et al., 2012; Bousmalis et al., 2016a; Kim et al., 2017b; Chang et al., 2019), us-ing adversarial and orthogonality losses (Liu et al., 2017; Li et al., 2018).”
Iterative Delexicalization for Improved Spoken Language Understanding (2019, methodology): “…parsing result for the user utterance x. Recurrent neural network parsers: RNN based parsers have been successfully applied for intent classification and slot labeling tasks in recent years [Mesnil et al., 2015; Hakkani-Tür et al., 2016; Liu and Lane, 2016; Kim et al., 2017a; Goo et al., 2018].”