Scatter Component Analysis: A Unified Framework for Domain Adaptation and Domain Generalization

By Muhammad Ghifary, David Balduzzi, W. Kleijn, Mengjie Zhang. 2015

483 citations recorded by the source corpus.

This paper addresses classification tasks on a particular target domain in which labeled training data are only available from source domains different from (but related to) the target. Two closely related frameworks, domain adaptation and domain generalization, are concerned with such tasks, where the only difference between those frameworks is the availability of the unlabeled target data: domain adaptation can leverage unlabeled target information, while domain generalization cannot. We propose Scatter Component Analyis (SCA), a fast representation learning algorithm that can be applied to both domain adaptation and domain generalization. SCA is based on a simple geometrical measure, i.e., scatter, which operates on reproducing kernel Hilbert space. SCA finds a representation that trades between maximizing the separability of classes, minimizing the mismatch between domains, and maximizing the separability of data; each of which is quantified through scatter. The optimization problem of SCA can be reduced to a generalized eigenvalue problem, which results in a fast and exact solution. Comprehensive experiments on benchmark cross-domain object recognition datasets verify that SCA performs much faster than several state-of-the-art algorithms and also provides state-of-the-art classification accuracy in both domain adaptation and domain generalization. We also show that scatter can be used to establish a theoretical generalization bound in the case of domain adaptation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Unified Deep Supervised Domain Adaptation and Generalization (2017, methodology): “1NN SVM LB UML [17] LRE-SVM [65] SCA [23] CCSA”
Detach and Adapt: Learning Cross-Domain Disentangled Deep Representation (2017, background): “GFK [8] JDA [22] SA [4] TJM [23] SCA [6] JGSA [33] DC [30] GR [5] CoGAN [20] ADDA [31] DRCN [7] ADGAN [28] CDRD”
Domain Adaptation by Joint Distribution Invariant Projections (2020, background): “A domain is a joint probability distribution P(x, y) defined on X × Y [6].”
A Fourier-based Framework for Domain Generalization (2021, background): “Early DG studies mainly follow the distribution alignment idea in domain adaptation by learning domaininvariant features via either kernel methods [29, 9] or domain-adversarial learning [24, 23, 38].”
Domain Generalization: A Survey (2021, methodology): “Therefore, to achieve invariance between source domains, one can learn a mapping function (e.g., a simple projection matrix [165] or a complex non-linear function modeled by deep neural networks [167]) with an objective of minimizing the moments of the transformed features between source domains,…”
Joint Geometrical and Statistical Alignment for Visual Domain Adaptation (2017, methodology): “Scatter component analysis (SCA) [9] takes the between and within class scatter of source domain into consideration.”
Reducing Domain Gap by Reducing Style Bias (2021, background): “A popular approach is learning a shared feature space across multiple source domains, for example, by minimizing Maximum Mean Discrepancy (MMD) [38, 16] or adversarial feature alignment [31, 32].”
Deep Domain Generalization With Structured Low-Rank Constraint (2018, background): “Most recently, domain generalization [16]–[23] has been well exploited to fight off the challenge through capturing”
Domain Generalization by Marginal Transfer Learning (2017, background): “…space) to compare distributions (Pan et al., 2011; Muandet et al., 2013; Maurer et al., 2013; Pentina and Lampert, 2014; Grubinger et al., 2015; Ghifary et al., 2017), generally to ﬁnd a projection in kernel space realizing a suitable compromise between matching of transformed marginal…”
Visual Domain Adaptation with Manifold Embedded Distribution Alignment (2018, methodology): “• 1NN, SVM, and PCA • Transfer Component Analysis (TCA) [26], which performs marginal distribution alignment • Geodesic Flow Kernel (GFK) [15], which performs manifold feature learning • Joint distribution alignment (JDA) [23], which adapts both marginal and conditional distribution • Transfer Joint”