The Need for Biases in Learning Generalizations

By Tom Michael Mitchell. 2007

708 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Experience Grounds Language (2020, background): “Models learn what they need to discriminate between potential labels, and it is unlikely that universal function approximators such as neural networks would ever reliably posit that people, events, and causality exist without being biased towards such solutions (Mitchell, 1980).”
The Natural Language Decathlon: Multitask Learning as Question Answering (2018, background): “When tasks are sufficiently related, they can provide an inductive bias [Mitchell, 1980] that forces models to learn more generally useful representations.”
Meta-learning with differentiable closed-form solvers (2018, methodology): “According to widely accepted deﬁnitions of learning (Mitchell, 1980) and meta-learning (Vilalta & Drissi, 2002; Vinyals et al., 2016), an algorithm is “learning to learn” if it can improve its learning skills with the number of experienced episodes (by progressively and dynamically modifying its…”
Relational inductive biases, deep learning, and graph networks (2018, background): “…inductive biases , in the form of speciﬁc architectural assumptions, which guide these approaches towards learning about entities and relations (Mitchell, 1980), which we, joining many others (Spelke et al., 1992; Spelke and Kinzler, 2007; Marcus, 2001; Tenenbaum et al., 2011; Lake et al.,…”
Encoder Based Lifelong Learning (2017, methodology): “Standard multi-task learning [4] aims to learn jointly from the data of the multiple tasks and uses inductive bias [17] in order to integrate the knowledge from the different domains in a single model.”
Expert Gate: Lifelong Learning with a Network of Experts (2016, methodology): “The objective is to use knowledge from different tasks, the so called inductive bias [25], in order to improve performance on individual tasks.”
Lifelong Generative Modeling (2017, background): “These assumptions are referred to as hypothesis spaces, biases or priors and aid the model in favoring one solution over another [82, 128].”
Knowledge Matters: Importance of Prior Information for Optimization (2013, background): “This prior information and hints given to the learner can be viewed as inductive bias for a particular task, an important ingredient to obtain a good generalization error (Mitchell, 1980).”
BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning (2018, background): “Multitask Learning aims to improve generalization of a certain task by learning from other tasks [6, 23].”
A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks (2018, background): “Although, (i) the selection of this diverse set of tasks, as well as, (ii) the control of the interactions between them are of great importance, a deeper understanding of (i) and (ii) is missing as highlighted in the literature (Caruana 1997; Mitchell 1980; Ruder 2017).”
Bias and Generalization in Deep Generative Models: An Empirical Study (2018, methodology): “We will refer to the process of producing q(x) from D as generalization, and any assumptions used when producing q(x) from D as inductive bias [12].”
Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision (2020, background): “mples and their labels. However, correlations can be spurious, in the sense that they may re ect dataset-specic biases or sampling artifacts, rather than intrinsic properties of the task of interest [41,67]. When spurious correlations do not hold in the test data, the model’s predictive performance ”
Meta-Q-Learning (2019, background): “Conceptually, the improved efficiency of a meta-learner comes from two things: (i) building a better inductive bias to initialize the learning (Schmidhuber et al., 1997; Baxter, 1995; 2000; Mitchell, 1980), or (ii) learning a better learning procedure (Bengio et al.”
Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods (2019, methodology): “The specification of H always comes with an inductive bias, which is indeed essential for learning from data (Mitchell, 1980).”
A Transdisciplinary Review of Deep Learning Research and Its Relevance for Water Resources Scientists (2017, background): “They concluded that, just like infants, deep networks have strong bias for shape over color, and that training can incrementally increase this “inductive bias” [Mitchell and Mitchell, 1980].”