MetaReg: Towards Domain Generalization using Meta-Regularization

By Yogesh Kumar Balaji, S. Sankaranarayanan, Ramalingam Chellappa. 2018

786 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Optimization & Training

Citations in the corpus

Learning Robust Global Representations by Penalizing Local Predictive Power (2019, methodology): “, 2017b; Balaji et al., 2018). More recently, Wang et al. (2019) demonstrated promising results on a number of benchmarks without using domain identifiers.”
Learngene: From Open-World to Your Learning Task (2021, background): “Although meta domain adaptation [2, 37] uses a lightweight model, it cannot continually learn new tasks in an open-world.”
SWAD: Domain Generalization by Seeking Flat Minima (2021, neutral): “MASF [14] 82.7 - - - - - DMG [33] 83.4 - - - 43.6 - MetaReg [15] 83.6 - - - 43.6 - ER [12] 85.3 - - - - - pAdaIN [47] 85.4 - - - - - EISNet [48] 85.8 - - - - - DSON [30] 86.”
Incremental Few-Shot Learning with Attention Attractor Networks (2018, methodology): “Truncated BPTT is a popular learning algorithm in many recent meta-learning approaches [2, 27, 7, 34, 3].”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “, 2019); and meta-learning-based approaches (Li et al., 2018a; Balaji et al., 2018; Dou et al., 2019).”
Domain Generalization Using a Mixture of Multiple Latent Domains (2019, methodology): “In addition, several methods have been proposed, such as a method that is based on meta learning (Li et al. 2017b; Balaji, Sankaranarayanan, and Chellappa 2018), one that uses domain-speciﬁc aggregation modules (D’Innocente and Caputo 2018), and a method that combines supervised learning and…”
SMIL: Multimodal Learning with Severely Missing Modality (2021, background): “Balaji et al. (Balaji, Sankaranarayanan, and Chellappa 2018) modeled the regularization function as an additional network learned through meta-learning to regularize model parameters.”
A Review of Single-Source Deep Unsupervised Visual Domain Adaptation (2020, background): “Cross-domain meta learning provides promising solutions to address these challenges by essentially learning more transferable representations [153, 192].”
Learning to Generate Novel Domains for Domain Generalization (2020, positive): “ This can be achieved by dividing source domains into meta-train and meta-test sets without overlapping, and training a model on the meta-train set such that the error on the meta-test set is reduced [26,2,10]. Similar to domain alignment methods, meta-learning methods still risk overtting since the”
iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients (2021, methodology): “This naive approximation is also adopted by (Luketina et al., 2016; Balaji et al., 2018; Nichol et al., 2018).”
Test-Time Training with Self-Supervision for Generalization under Distribution Shifts (2019, background): “…multiple environment distributions, some of which are available during training (source), while others are used for testing (target) (Li et al., 2018; Shankar et al., 2018; Muandet et al., 2013; Balaji et al., 2018; Ghifary et al., 2015; Motiian et al., 2017; Li et al., 2017a; Gan et al., 2016).”
Optimizing Millions of Hyperparameters by Implicit Differentiation (2019, methodology): “Various works use an identity approximation to the inverse, which is equivalent to 1 -step unrolled diﬀerentiation [24, 14, 33, 10, 8, 34, 35].”
Towards Out-Of-Distribution Generalization: A Survey (2021, background): “…(MAML) for DA, which introduces the concept of ”episodes” in the training phase and greatly inﬂuence the research of meta learning for DG. Li et al. [72] ﬁrst apply meta-learning to DG and some improvements are made in the following works [73], [74], [75], [76], [77], [78], [79], [80], [81].”
Revisiting Meta-Learning as Supervised Learning (2020, methodology): “…2017; Wang and Hebert, 2016), optimization (Li and Malik, 2017; Wichrowska et al., 2017), active learning (Bachman et al., 2017), transfer learning (Balaji et al., 2018; Ying et al., 2018), unsupervised learning (Metz et al., 2018; Edwards and Storkey, 2017), etc. Specifically, metalearning has…”
Learning to Balance Specificity and Invariance for In and Out of Domain Generalization (2020, methodology): “rformance, though we believe this may be dataset-specic. Domain Specic Masks for Generalization 9 Table 1: Out of Domain Accuracy (%) on DomainNet ( O = 0.1) We were unable to optimize the MetaReg [3] objective with Adam [23] as the optimizer and therefore, we also include comparisons with Aggregate”
Domain Generalization via Model-Agnostic Learning of Semantic Features (2019, methodology): “Promising performance has been demonstrated by deriving the loss from a task error [26], a classifier regularizer [1], or a predictive feature-critic module [31].”