A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling

By Jonathan Baxter. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Better Mixing via Deep Representations (2012, background): “L G ] 1 8 ing work such as multi-task learning algorithms (Caruana, 1995; Baxter, 1997; Collobert and Weston, 2008) and learning algorithms involving parameter sharing (Lang and Hinton, 1988; LeCun, 1989).”
An Overview of Multi-Task Learning in Deep Neural Networks (2017, methodology): “Most approaches in the history of MTL have focused on the scenario where tasks are drawn from the same distribution [Baxter, 1997].”
Exploring Shared Structures and Hierarchies for Multiple NLP Tasks (2018, neutral): “This is a basic shared strategy and an efﬁcient approach to ﬁght overﬁt-ting (Baxter 1997).”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, background): “Parameter sharing (Lang and Hinton, 1988; LeCun, 1989; Lang and Hinton, 1988; Caruana, 1993; Baxter, 1995, 1997) is an old neural network technique for increasing statistical power: if a parameter is used in N times more contexts (different tasks, different parts of the input, etc.) then it may be…”
Taskonomy: Disentangling Task Transfer Learning (2018, methodology): “transfer taxonomy using BIP (Binary Integer Program). to map a hypothesis for one task onto a hypothesis for another [7], through information-based approaches [60], or through modeling inductive bias [6]. For these guarantees, learning theoretic approaches usually rely on intractable computations, o”
A Convex Formulation for Learning Task Relationships in Multi-Task Learning (2010, background): “ and some existing multi-task learning methods. Experiments conducted on a toy problem as well as several benchmark data sets demonstrate the effectiveness of MTRL. 1 Introduction Multi-task learning [9, 4, 24] is a learning paradigm which seeks to improve the generalization performance of a learnin”
Efficient Representations for Lifelong Learning and Autoencoding (2014, background): “Baxter [7, 6] developed some of the earliest foundations for transfer learning, by providing sample complexity results for achieving low average error in such settings.”
A theory of transfer learning with applications to active learning (2012, background): “Essentially, [Bax97] sets up a hierarchical Bayesian model, in which (in our notation) θ? is a random variable with known distribution (hyper-prior), but otherwise the specialization of Baxter’s setting to the pattern recognition problem is essentially identical to our setup above.”
A No-Free-Lunch Theorem for MultiTask Learning (2020, background): “Following this spirit, many theoretical works on the subject are concerned with bounds on average risk across tasks (Baxter, 1997; Ando and Zhang, 2005; Maurer, Pontil and Romera-Paredes, 2013; Pentina and Lampert, 2014; Yang, Hanneke and Carbonell, 2013; Maurer, Pontil and Romera-Paredes, 2016),…”
Multi-task learning for learning to rank in web search (2009, background): “The second family of approaches assume that common structures can also be represented by super-features shared among multiple tasks [1,4].”
Multi-task warped Gaussian process for personalized age estimation (2010, background): “Multi-task learning [3, 2, 28] is a learning paradigm which seeks to improve the generalization performance of a learning task with the help of some other related tasks.”
Heterogeneous-Neighborhood-based Multi-Task Local Learning Algorithms (2013, methodology): “In this case, we can leverage the useful information from other related tasks to help improve the performance which matches the philosophy of multi-task learning [8, 4, 16].”
Learning Multiple Tasks with Kernel Methods (2005, methodology): “…of the sensitivity of our approach to the choice of the parameter λ is an open research direction which may also lead to a better understanding of the effects of increasing the number of tasks on the generalization performance as discussed in (Baxter, 1997, 2000; Ben-David and Schuller, 2003).”
Learning Visual Representations using Images with Captions (2007, background): “Transfer and multi-task learning have a relatively long history in machine learning [20, 4, 16, 1].”
Transitive Transfer Learning (2015, background): “Multi-task Learning algorithms simultaneously learn several tasks together and mutually enhance the classification results of each task [2, 4, 5].”
Semi-Supervised Multi-Task Regression (2009, background): “On the other hand, multi-task learning [8–10] seeks to improve the learning performance of one task with the help of some related tasks.”