Online Methods for Multi-Domain Learning and Adaptation

By Mark Dredze, Koby Crammer. 2008

90 citations recorded by the source corpus.

NLP tasks are often domain specific, yet systems can learn behaviors across multiple domains. We develop a new multi-domain online learning framework based on parameter combination from multiple classifiers. Our algorithms draw from multi-task learning and domain adaptation to adapt multiple source domain classifiers to a new target domain, learn across multiple similar domains, and learn across a large number of disparate domains. We evaluate our algorithms on two popular NLP domain adaptation tasks: sentiment classification and spam filtering.

Subfields: Artificial Intelligence, Information Systems, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Sociology and Political Science

Citations in the corpus

Unsupervised Model Adaptation for Continual Semantic Segmentation (2020, methodology): “This model adaptation setting been explored for non-deep models (Dredze and Crammer 2008; Jain and Learned-Miller 2011; Wu 2016), but these works cannot be extended to semantic segmentation tasks.”
Bayesian Multitask Learning with Latent Hierarchies (2009, background): “Dredze and Crammer [8] describe an online approach for dealing with the many-domains problem, sharing information across domains via confidence-weighted classifiers.”
Domain Adaptation with Coupled Subspaces (2011, background): “The sentiment prediction task [7, 28, 12] consists of reviews of four different types of products: books, DVDs, electronics, and kitchen appliances from Amazon.”
Multi-Domain Learning: When Do Domains Matter? (2012, methodology): “Instead, multi-domain learning (MDL) can take advantage of these domain labels to improve learning (Daumé III, 2007; Dredze and Crammer, 2008; Arnold et al., 2008; Finkel and Manning, 2009; Zhang and Yeung, 2010; Saha et al., 2011).”
Multi-domain learning by confidence-weighted parameter combination (2010, methodology): “Instead, we seek a similar solution to Dredze et al. (2008): we simplify the objective to obtain a closed form solution.”
A survey of multi-source domain adaptation (2015, background): “[23] showed how domain adaptation for parsing is difficult when annotation guidelines differ for different domains.”
Domain adaptation for object recognition: An unsupervised approach (2011, background): “There can also be scenarios where we have multiple domains in source and target [30, 17].”
Multi-task Domain Adaptation for Sequence Tagging (2016, unknown): “Another line sought for feature weight tying (Dredze and Crammer, 2008; Daum´e III, 2007, 2009; Finkel and Manning, 2009; Kumar et al., 2010; Dredze et al., 2010) to transfer the learned feature weights across domains.”
Collaborating between Local and Global Learning for Distributed Online Multiple Tasks (2015, methodology): “In online multi-domain learning [16], it first uses single task online learning method to learn a model for each task separately, and then proposes a method to combined all these models into a synthesized model.”
Sequential Unsupervised Domain Adaptation through Prototypical Distributions (2020, methodology): “This learning setting for domain adaptation has been explored for non-deep models [23–25].”
Learning Sentence Embeddings with Auxiliary Tasks for Cross-Domain Sentiment Classification (2016, background): “Domain Adaptation: Domain adaptation is a general problem in NLP and has been well studied in recent years (Blitzer et al., 2006; Daumé III, 2007; Jiang and Zhai, 2007; Dredze and Crammer, 2008; Titov, 2011; Yu and Jiang, 2015).”
Multi-Task Transfer Learning for Weakly-Supervised Relation Extraction (2009, methodology): “…decade ago (Thrun, 1996; Caruana, 1997), its application in natural language processing is still a relatively new territory (Blitzer et al., 2006; Daume III, 2007; Jiang and Zhai, 2007a; Arnold et al., 2008; Dredze and Crammer, 2008), and its application in relation extraction is still unexplored.”
Incremental Evolving Domain Adaptation (2016, methodology): “The online adaptation method introduced in [28] learns a single stationary target distribution from streaming unlabeled observations.”