Frustratingly Easy Domain Adaptation

By Hal Daumé Iii. 2007

1,845 citations recorded by the source corpus.

We describe an approach to domain adaptation that is appropriate exactly in the case when one has enough “target” data to do slightly better than just using only “source” data. Our approach is incredibly simple, easy to implement as a preprocessing step (10 lines of Perl!) and outperforms stateof-the-art approaches on a range of datasets. Moreover, it is trivially extended to a multidomain adaptation problem, where one has data from a variety of different domains.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Revisiting Few-sample BERT Fine-tuning (2020, methodology): “Pre-trained weight decay adapts this method for fine-tuning pre-trained models [3, 5] by subtracting λ(w − ŵ) from the objective, where ŵ is the pre-trained parameters.”
A theory of learning from different domains (2010, methodology): “Although these bounds have a more limited scope than ours, they can sometimes yield useful results even when the optimal predictors for each task (or domain in the case of Daumé 2007) are quite different (Baxter 2000; Ando and Zhang 2005).”
Marginalized Denoising Autoencoders for Domain Adaptation (2012, background): “Examples are computational biology (Liu et al., 2008), natural language processing (Daume III, 2007; McClosky et al., 2006) and computer vision (Saenko et al., 2010).”
Dataset Distillation (2018, background): “Extensive prior work has been proposed to adapt models to new tasks and datasets (Daume III, 2007; Saenko et al., 2010).”
Deep Domain Confusion: Maximizing for Domain Invariance (2014, methodology): “To test this hypothesis, for one of the Amazon → Webcam splits we apply a simple domain adaptation baseline introduced by Daum ´ e III [10] to compute test accuracy for the target domain.”
Feature hashing for large scale multitask learning (2009, methodology): “Personalization (Daum´e, 2007) introduced a very simple but strikingly effective method for multitask learning.”
Compact Bilinear Pooling (2015, methodology): “Our compact bilinear method, on the other hand, requires far fewer parameters in the classification layer, potentially reducing the risk of over-fitting, and performing better in few shot learning scenarios [12], or domain adaptation [11] scenarios.”
Efficient Learning of Domain-invariant Image Representations (2013, methodology): “Several SVMbased approaches have been proposed for image domain adaptation, including: weighted combination of source and target SVMs and transductive SVMs applied to adaptation in [21]; the feature replication method of [17]; Adaptive SVM [8, 9], where the source model parameters are adapted by add”
An embarrassingly simple approach to zero-shot learning (2015, background): “The aim of domain adaptation is to learn a function from data in one domain, so that it can be successfully applied to data from a different domain [4, 8, 19].”
Latent Multi-Task Architecture Learning (2017, background): “task learning algorithms such as hard parameter sharing [7], low supervision [25], and cross-stitch networks [21], as well as transfer learning algorithms such as frustratingly easy domain adaptation [9]. Moreover, we study what task properties predict gains, and what properties correlate with learn”
Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview (2019, background): “The general phenomenon of biased predictive models is perhaps not recent in NLP: the community has long worked on the domain adaptation problem Jiang and Zhai (2007); Daume III (2007): models ﬁt on newswire data do not perform well on social media and other text types.”
BilBOWA: Fast Bilingual Distributed Representations without Word Alignments (2014, background): “Although several techniques exist that can learn to map hand-crafted features from one domain to another [4, 6, 17], it is in general non-trivial to come up with good features which generalize well across tasks, and even harder across different languages.”
A Comprehensive Survey on Transfer Learning (2019, neutral): “A number of transfer learning approaches, for example, FAM [64] and the variant of KMM [5], are compared with each other.”