A Unified Perspective on Multi-Domain and Multi-Task Learning

By Yongxin Yang, Timothy M. Hospedales. 2014

167 citations recorded by the source corpus.

In this paper, we provide a new neural-network based perspective on multi-task learning (MTL) and multi-domain learning (MDL). By introducing the concept of a semantic descriptor, this framework unifies MDL and MTL as well as encompassing various classic and recent MTL/MDL algorithms by interpreting them as different ways of constructing semantic descriptors. Our interpretation provides an alternative pipeline for zero-shot learning (ZSL), where a model for a novel class can be constructed without training data. Moreover, it leads to a new and practically relevant problem setting of zero-shot domain adaptation (ZSDA), which is the analogous to ZSL but for novel domains: A model for an unseen domain can be generated by its semantic descriptor. Experiments across this range of problems demonstrate that our framework outperforms a variety of alternatives.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Predicting Deep Zero-Shot Convolutional Neural Networks Using Textual Descriptions (2015, background): “[31] considered unified formulation of domain adaptation and multi-task learning where they combine different domains using a dot-product operator.”
Cross-Stitch Networks for Multi-task Learning (2016, background): “The term multi-task learning (MTL) itself has been broadly used [2, 14, 28, 42, 54, 55] as an umbrella term to include representation learning and selection [4, 13, 31, 37], transfer learning [39, 41, 56] etc.”
Learning to Compare: Relation Network for Few-Shot Learning (2017, methodology): “When applied to ZSL our architecture is related to methods that learn to align images and category embeddings and perform recognition by predicting if an image and category embedding pair match [11, 3, 43, 46].”
An embarrassingly simple approach to zero-shot learning (2015, background): “Recent works exploring this direction are [18, 37].”
Learning to Generalize: Meta-Learning for Domain Generalization (2017, neutral): “Multi-Domain Learning (MDL) MDL addresses training a single model that is effective for multiple known domains (Daumé 2007; Yang and Hospedales 2015; Rebuff, Bilen, and Vedaldi 2017; Bilen and Vedaldi 2017; Zhao et al. 2017).”
Meta-Learning in Neural Networks: A Survey (2020, methodology): “, methods that group tasks into clusters [291] or subspaces [292].”
Domain Adaptation for Visual Applications: A Comprehensive Survey (2017, positive): “The Unifying Multi-Domain Multi-Task Learning [247], is a Neural Network framework that can be flexibly applied to multi-task, multidomain and zero-shot learning and even to zero-shot domain adaptation.”
Deep Multi-task Representation Learning: A Tensor Factorisation Approach (2016, methodology): “S is a K × T size matrix, where T is the number of tasks, and K is the number of latent tasks [Kumar and Daum´e III, 2012] or the dimension of task coding [Yang and Hospedales, 2015].”
Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitation (2018, background): “That is why more and more studies use synthetic data to train imitation learning methods in a virtual environment [6,34,16,41].”
Multi-Domain Adversarial Learning (2019, methodology): “The domain-dependent use of these parameters can be learned, e.g. using domain-guided dropout (Xiao et al., 2016), or based on prior knowledge about domain semantic relationships (Yang & Hospedales, 2015).”
Learning a Deep Embedding Model for Zero-Shot Learning (2016, methodology): “For fair comparison, we implement the models in [10, 43, 46, 24] on AwA and CUB with Inception-V2 as the CNN subnet as in our model and [34].”
Generalizing Across Domains via Cross-Gradient Training (2018, background): “| While Yang & Hospedales (2015) addressed a similar setting, they assumed a speciﬁc geometry characterizing the domain, and performed kernel regression in this space.”
iLab-20M: A Large-Scale Controlled Object Dataset to Investigate Deep Learning (2016, background): “Some recent works have attempted to show that there is a real benefit of these datasets in transferring knowledge to large-scale natural scene datasets [26, 67].”
Generalized Zero-Shot Learning with Deep Calibration Network (2018, methodology): “Comparison methods We choose to compare with many competitive or representative methods, including shallow methods DAP [32], ALE [1], SJE [3], ESZSL [42], ConSE [36], SynC [8], EXEM [9], ZSKL [57], and deep methods DeViSE [15], CMT [47], MTMDL [55], DEM [58].”
A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts (2017, background): “The core of such approaches is to (a) learn a mapping function from the visual feature space to the semantic space [41, 11, 12], or conversely [51, 39], (b) or jointly learn the embedding function between the visual and semantic space through a latent space [46, 22, 3, 36, 1].”
Episodic Training for Domain Generalization (2019, background): “Multi-Domain Learning (MDL) MDL aims to learn several domains simultaneously using a single model [3, 28, 29, 39].”
Zero-shot Domain Adaptation without Domain Semantic Descriptors (2018, methodology): “Note that this formulation is similar to the model described by Yang and Hospedales [42] although zd represents an observed semantic descriptor characterizing a domain in their work [42].”
Towards Universal Object Detection by Domain Attention (2019, background): “Multi-domain learning (MDL) [20, 35, 24, 59, 19, 5] improves on this, by sharing parameters across various domains, and adding small domainspecific layers.”
Trace Norm Regularised Deep Multi-Task Learning (2016, methodology): “In this section, we will brieﬂy review some studies in this area.”