Domain Adaptation by Mixture of Alignments of Second-or Higher-Order Scatter Tensors

By Piotr Koniusz, Yusuf Tas, Fatih Murat Porikli. 2016

125 citations recorded by the source corpus.

In this paper, we propose an approach to the domain adaptation, dubbed Second-or Higher-order Transfer of Knowledge (So-HoT), based on the mixture of alignments of second-or higher-order scatter statistics between the source and target domains. The human ability to learn from few labeled samples is a recurring motivation in the literature for domain adaptation. Towards this end, we investigate the supervised target scenario for which few labeled target training samples per category exist. Specifically, we utilize two CNN streams: the source and target networks fused at the classifier level. Features from the fully connected layers fc7 of each network are used to compute second-or even higher-order scatter tensors, one per network stream per class. As the source and target distributions are somewhat different despite being related, we align the scatters of the two network streams of the same class (within-class scatters) to a desired degree with our bespoke loss while maintaining good separation of the between-class scatters. We train the entire network in end-to-end fashion. We provide evaluations on the standard Office benchmark (visual domains) and RGB-D combined with Caltech256 (depth-to-rgb transfer). We attain state-of-the-art results.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

Few-Shot Adversarial Domain Adaptation (2017, methodology): “[27] uses two CNN streams, for source and target, fused at the classifier level.”
Unified Deep Supervised Domain Adaptation and Generalization (2017, methodology): “We also report the classification results of the SDA algorithm presented in [39] and [33].”
HGNet: Hybrid Generative Network for Zero-Shot Domain Adaptation (2020, background): “And recent works attempt to apply DA strategy in computer vision like image classiﬁcation [12, 18], object segmentation [29, 36, 26] and image caption [3].”
Partial Adversarial Domain Adaptation (2018, background): “Supervised domain adaptation [11,21,22,23] exploits a few labeled data in the target domain.”
Beyond Sharing Weights for Deep Domain Adaptation (2016, methodology): “A similar idea was followed in [44] to minimize the difference in second order feature statistics using the CORAL loss [45] and to relate higher order feature statistics in [46].”
Small Sample Learning in Big Data Era (2018, background): “The latest work focused on the large domain shifts between source and target datasets in (Koniusz et al., 2018), and a new dataset called open museum identification challenge (Open MIC) was released.”
Adversarial Domain Adaptation With Prototype-Based Normalized Output Conditioner (2020, methodology): “Previous methods exploit maximum mean discrepancy (MMD) [1], [8] or other distribution statistics like central moments [9]–[11] for domain adaptation.”
Hallucinating IDT Descriptors and I3D Optical Flow Features for Action Recognition With CNNs (2019, methodology): “We perform translation of the I3D output into IDT-based BoW/FV descriptors thus enforcing I3D to implicitly learn IDT which coregularizes I3D which resembles Domain Adaptation (DA) methods: a source network co-regularizes a target network [34, 58, 35, 25, 24, 42] by the alignment of feature statisti”
Multi-Domain Adversarial Learning (2019, result): “, 2012) yield worse results, as already noted by (Koniusz et al., 2016).”
Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift (2018, methodology): “Besides of these two mainstreams, there are diverse methods to learn domain-invariant features: semi-supervised method [42], domain reconstruction [14], duality [19], alignments [9] [50] [44], manifold learning [15], tensor methods [24][31], etc. Domain adaptation with multiple sources.”
A Balanced and Uncertainty-aware Approach for Partial Domain Adaptation (2020, background): “ed into three main groups. The rst group aims to minimize the domain discrepancy by matching dierent statistic moments like maximum mean discrepancy (MMD) [40,23,26] and higher-order moment matching [36,46,18]. The second that is widely used introduces a domain discriminator and exploits the idea of”