Algorithmic Stability and Meta-Learning

By Andreas Maurer. 2005

118 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Yu Zhang
2. Han-Jia Ye
3. Qiang Yang
4. Mingsheng Long
5. Dacheng Tao
6. Yoshua Bengio
7. Wei-Lun Chao
8. Bernt Schiele

Citations in the corpus

• Adaptive Gradient-Based Meta-Learning Methods (2019, background): “While theoretical LTL has a long history (Baxter, 2000; Maurer, 2005; Pentina & Lampert, 2014), there has recently been an effort to understand GBML in particular.”
• Multi-Task Learning and Algorithmic Stability (2015, methodology): “Moreover, stability has been applied in (Maurer 2005; Kuzborskij and Orabona 2013) to analyze transfer learning algorithms which are related to multi-task learning but have different settings since transfer learning aims to improve the performance of only the target task by leveraging the…”
• Incremental Learning-to-Learn with Statistical Guarantees (2018, methodology): “, μT , has been subject of thorough analysis in the literature [6, 19, 23].”
• Revisiting Meta-Learning as Supervised Learning (2020, methodology): “a unifying view of meta-learning through revisiting and strengthening the connection to supervised learning (Thrun and Pratt, 1998; Baxter, 2000; Maurer, 2005; Wang, 2018; Larochelle, 2018; Finn, 2018).”
• Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory (2017, background): “Other recent theoretical approaches to meta or multitask learning (e.g. Maurer (2005; 2009); Ruvolo & Eaton (2013); Maurer et al. (2016); Alquier et al. (2017)) provide increasingly general bounds but have not led to practical algorithms for neural networks.”
• Transfer bounds for linear feature learning (2009, methodology): “Instead of using covering numbers (Maurer 2005) proposes stability arguments to bound the second term in the decomposition.”
• Ensemble contextual bandits for personalized recommendation (2014, methodology): “Meta learning has been widely used in algorithm selection [16, 19].”
• Regret Bounds for Lifelong Learning (2016, background): “Although a substantial amount of work has been done on the theoretical study of learning-to-learn (Baxter, 2000; Maurer, 2005; Pentina and Lampert, 2014; Maurer et al., 2016), up to our knowledge there is no analysis of the statistical performance of lifelong learning algorithms.”
• A Sample Complexity Separation between Non-Convex and Convex Meta-Learning (2020, background): “There is a rich history of theoretical analysis of learning-tolearn (Baxter, 2000; Maurer, 2005; Maurer et al., 2016).”
• Provable Guarantees for Gradient-Based Meta-Learning (2019, methodology): “Finally, we also provide an online-to-batch conversion result for task-averaged regret that implies good generalization guarantees when any of the variants of Ephemeral are run in the distributional LTL setting.”
• Lifelong Learning with Non-i.i.d. Tasks (2015, methodology): “In contrast to previous works on lifelong learning [2, 5, 8] we omit the assumption that the observed tasks are independently and identically distributed.”
• Transfer Learning via Learning to Transfer (2018, background): “Lifelong Learning Assuming a new learning task to lie in the same environment as training tasks, learning to learn (Thrun & Pratt, 1998) or meta-learning (Maurer, 2005; Finn et al., 2017; Al-Shedivat et al., 2018) transfers the knowledge shared among training tasks to the new task.”
• Learning from Multiple Sources (2006, background): “The framework examined here can also be viewed in the context of multi-task learning, or as a type of transfer learning (Baxter, 1995; Ben-David, 2003; Maurer, 2005).”
• Learning To Learn Around A Common Mean (2018, methodology): “In literature, the LTL problem has been almost exclusively considered in the setting in which the tasks are given in one batch [4, 19, 20, 21, 22, 24], as opposed to sequentially.”
• Learning-to-Learn Stochastic Gradient Descent with Biased Regularization (2019, methodology): “The performance of the selected inner algorithm is measured by the transfer risk (Baxter, 2000; Maurer, 2005), that is, the average risk of the algorithm, trained on a random dataset from the same environment.”