Regret Bounds for Lifelong Learning

By Pierre Alquier, The Tien Mai, Massimiliano Pontil. 2016

61 citations recorded by the source corpus.

We consider the problem of transfer learning in an online setting. Different tasks are presented sequentially and processed by a within-task algorithm. We propose a lifelong learning strategy which refines the underlying data representation used by the within-task algorithm, thereby transferring information from one task to the next. We show that when the within-task algorithm comes with some regret bound, our strategy inherits this good property. Our bounds are in expectation for a general loss function, and uniform for a convex loss. We discuss applications to dictionary learning and finite set of predictors. In the latter case, we improve previous $O(1/\sqrt{m})$ bounds to $O(1/m)$ where $m$ is the per task sample size.

Subfields: Artificial Intelligence, Computational Mechanics, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Management Science and Operations Research

Citations in the corpus

Adaptive Gradient-Based Meta-Learning Methods (2019, background): “Batch-setting transfer risk bounds have been an important motivation for studying LTL via online learning [2, 34, 19].”
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent (2020, background): “…lifelong learning and meta-learning from a statistical learning theory perspective, they define and provide regret bounds for lifelong learning (Alquier et al., 2017), or study the sample complexity and convergence of meta-learning algorithms (Denevi et al. 2018a, Du et al. 2020, Ji et al.…”
Online Meta-Learning (2019, neutral): “Recent and concurrent works have also studied algorithms related to MAML and its first order variants using theoretical tools from online learning literature (Alquier et al., 2016; Denevi et al., 2019; Khodak et al., 2019).”
Few-Shot Learning via Learning the Representation, Provably (2020, background): “Lastly, we remark that there are analyses for other representation learning schemes (Arora et al., 2019, McNamara and Balcan, 2017, Galanti et al., 2016, Alquier et al., 2016, Denevi et al., 2018).”
Incremental Learning-to-Learn with Statistical Guarantees (2018, background): “Perhaps most related to our work is (Alquier et al., 2017), where the authors consider a general PACBayesian approach to lifelong learning based on the exponentially weighted aggregation procedure.”
Multi-Step Model-Agnostic Meta-Learning: Convergence and Improved Algorithms (2020, methodology): “…meta-initialization algorithms such as MAML, meta-regularization approaches aim to learn a good bias for a regularized empirical risk minimization problem for intra-task learning (Alquier et al., 2016; Denevi et al., 2018b,a, 2019; Rajeswaran et al., 2019; Balcan et al., 2019; Zhou et al., 2019).”
Provable Continual Learning via Sketched Jacobian Approximations (2021, unknown): “Some recent theoretical developments include (Yin et al., 2020; Bennani et al., 2020; Alquier et al., 2017; Doan et al., 2021).”
Learning-to-Learn Stochastic Gradient Descent with Biased Regularization (2019, methodology): “, 2018; Ravi & Larochelle, 2017) and the theoretical challenge of statistically principled and efficient solutions (Alquier et al., 2017; Balcan et al., 2015; Maurer et al., 2016; Pentina & Lampert, 2014; Denevi et al., 2018a;b; Gupta & Roughgarden, 2017).”
On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms (2019, background): “A closely related problem to this setting is the lifelong learning framework (Balcan et al., 2015; Alquier et al., 2017).”
Efficient Meta Learning via Minibatch Proximal Update (2019, background): “Alternative to meta-initialization, the meta-regularization approaches have gained recent interest mostly due to their provable strong guarantees on statistical learning and computation efﬁciency [24, 25, 26, 27, 21].”
An Online Learning Approach to Interpolation and Extrapolation in Domain Generalization (2021, background): “, 2015; Hazan, 2016), for which there already exist provable regret bounds and efficiency guarantees (Balcan et al., 2015; Alquier et al., 2017).”
Learning To Learn Around A Common Mean (2018, background): “Most related to our work are [1], where the authors consider a general PAC-Bayesian approach to LTL and [12], which considers the problem of learning a linear representation shared by the tasks.”
Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory (2017, background): “Other recent theoretical approaches to meta or multitask learning (e.g. Maurer (2005; 2009); Ruvolo & Eaton (2013); Maurer et al. (2016); Alquier et al. (2017)) provide increasingly general bounds but have not led to practical algorithms for neural networks.”
Provable Guarantees for Gradient-Based Meta-Learning (2019, methodology): “…there are many theoretical results for learning useful data representations (Ruvolo & Eaton, 2013; Pentina & Lam-pert, 2014; Balcan et al., 2015; Alquier et al., 2017); the PAC-Bayesian result of Pentina & Lampert (2014) can also be used for regularization-based parameter transfer, which we…”
A Continual Learning Survey: Defying Forgetting in Classification Tasks (2019, background): “The inﬂuence of dropout [43] and different activation functions on forgetting have been studied empirically for two sequential tasks [31] , and several works studied task incremental learning from a theoretical perspective [44], [45], [46], [47].”