The Lack of A Priori Distinctions Between Learning Algorithms

By M. Kauffman, San Mateo, Edward C. Prescott. 1996

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Meta-Learning in Neural Networks: A Survey (2020, neutral): “This perspective [29], [30], [31] views meta-learning as a tool to manage the ‘no free lunch’ theorem [32] and improve gen-”
Sacrificing Accuracy for Transparency in Recidivism Risk Assessment: The Impact of Classification Method on Predictive Performance (2016, methodology): “The formula for SAR is: (ACC + AUC + (1 – RMSE))/3 (Caruana, Niculescu-Mizil, Crew, & Ksikes, 2004).”
Metaheuristic design of feedforward neural networks: A review of two decades of research (2017, background): “Moreover, Wolpert [92] introduced NFL for optimization algorithm to answer the question, “How does the set of problems F1 ⊂ F for which algorithm a1 performs better than algorithm a2 compares to the set F2 ⊂ F for which the reverse is true.””
On Value Functions and the Agent-Environment Boundary (2019, methodology): “We provide an argument below, inspired by that of the No Free Lunch theorem [Wolpert, 1996].”
Shallow vs. Deep Sum-Product Networks (2011, background): “…is no universally optimal choice of parameterization or family of functions (or " architecture " ), as demonstrated by the no-free-lunch results [37], it may be the case that some architectures are appropriate (or inappropriate ) for a large class of learning tasks and data distributions, such…”
Gradient Starvation: A Learning Proclivity in Neural Networks (2020, background): “The no free lunch theorem [93, 102] states that “learning is impossible without making assumptions about training and test distributions”.”
Reinforcement Learning with Self-Modifying Policies (1998, background): “There is no miraculous universal learning algorithm that will perform well in arbitrary environments [1, 2, 3].”
Shifting Inductive Bias with Success-Story Algorithm, Adaptive Levin Search, and Incremental Self-Improvement (1997, background): “This implies that inductive transfer and “generalization” are almost always impossible — see, e.g., Schmidhuber (1997a); for related results see Wolpert (1996).”
Hands-On Bayesian Neural Networks—A Tutorial for Deep Learning Users (2020, background): “Third , the no-free-lunch theorem for machine learning [31] can be interpreted as stating that any supervised learning algorithm includes some implicit prior.”
Applied Predictive Modeling (2013, unknown): “The “No Free Lunch”Theorem (Wolpert 1996) argues that,”
Lifelong Machine Learning Systems: Beyond Learning Algorithms (2013, background): “Since that time, the AI community has come to accept the futility of searching for a universal machine learning algorithm (Wolpert 1996).”
An Introduction to Machine Learning (2017, methodology): “The no-free-lunch theorem was published by Wolpert [100].”
An up-to-date comparison of state-of-the-art classification algorithms (2017, result): “This argument is in accordance with the No-Free-Lunch theorem ( Wolpert, 1996 ), which states that the best classiﬁer will not be the same for all the data sets.”
Prototype-based Neural Network Layers: Incorporating Vector Quantization (2018, background): “This observation seems to be plausible under the scope of the No-Free-Lunch theorem [69].”