No free lunch theorems for optimization

By David Hilton Wolpert, William G. Macready. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Evaluation & Benchmarks

Citations in the corpus

Scaling Laws for the Few-Shot Adaptation of Pre-trained Image Classifiers (2021, background): “According to the no free lunch theorem (Wolpert & Macready, 1997), not if we consider all possible data-generating distributions.”
Hyper-Parameter Optimization: A Review of Algorithms and Applications (2020, background): “The no free lunch theorem suggests that the computational cost for any optimization problem is the same for all problems and no solution offers a shortcut (Wolpert and Macready, 1997; Igel, 2014).”
Survey on Automated Machine Learning (2019, methodology): “According to the no free lunch theorem (Wolpert and Macready, 1997) it is impossible that a single optimization algorithm is universally superior to any other optimization algorithm.”
Neural Combinatorial Optimization with Reinforcement Learning (2016, background): “The difﬁculty in applying existing search heuristics to newly encountered problems - or even new instances of a similar problem - is a well-known challenge that stems from the No Free Lunch theorem (Wolpert & Macready, 1997).”
Generalisation in humans and deep neural networks (2018, neutral): “Our results, based on 82,880 psychophysical trials under well-controlled lab conditions, demonstrate that there are still marked differences in the way humans and current DNNs process (6)The no free lunch theorem [62] states that better performance on some input is necessarily accompanied by worse p”
Universal Intelligence: A Definition of Machine Intelligence (2007, background): “Some, such as Edmonds (2006), argue that universal intelligence is impossible due to Wolpert’s so called ‘‘No Free Lunch’’ theorem (Wolpert and Macready 1997).”
Exploring protein fitness landscapes by directed evolution (2009, neutral): “As with any optimization problem, the structure of the objective function (the fitness landscape) influences the effectiveness of a search strateg”
Occam's razor is insufficient to infer the preferences of irrational agents (2017, methodology): “A No Free Lunch result analogous to theirs applies in our case; both resemble the No Free Lunch theorems for optimisation [Wolpert and Macready, 1997].”
Reinforcement Learning with a Corrupted Reward Channel (2017, positive): “Similar to the No Free Lunch theorems for optimisation [Wolpert and Macready, 1997], the following theorem for CRMDPs says that without some assumption about what the reward corruption can look like, all agents are essentially lost.”
Similarity measures for identifying material parameters from hysteresis loops using inverse analysis (2018, background): “It is worth to note that different optimization techniques which appear to solve some inverse analyses well, may struggle on other particular inverse problems as principle of the No Free Lunch theorem [25].”
Optimizing connection weights in neural networks using the whale optimization algorithm (2016, methodology): “According to the no-free-lunch theorem (NFL), there is no superior optimization algorithm in all optimization problems (Wolpert and Macready 1997; Ho and Pepyne 2002).”
Gaussian Processes for history-matching: application to an unconventional gas reservoir (2017, background): “We should also note that as per the “no free lunch theorem” for optimization and search problems [72], finding a universal history-matching approach is theoretically impossible [8].”
An effective hybrid genetic algorithm and tabu search for flexible job shop scheduling problem (2016, methodology): “And no algorithm can solve all types of optimization problems based on the No Free Lunch Theorem (Wolpert and Macready, 1997).”