The Optimizer's Curse: Skepticism and Postdecision Surprise in Decision Analysis

By James E. Smith, Robert L. Winkler. 2006

328 citations recorded by the source corpus.

Subfields: Artificial Intelligence, General Decision Sciences, Management Science and Operations Research, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Decision Sciences

In Management Science and Operations Research

In Game & Decision Theory

Citations in the corpus

Taming the Noise in Reinforcement Learning via Soft Updates (2015, background): “This problem is akin to the “winner’s curse” in auctions [6, 7, 8, 9].”
Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Critics (2020, background): “It is known as the max-operator bias in statistics (D’Eramo et al., 2017) and as the ”winner’s curse” in economics (Smith & Winkler, 2006; Thaler, 2012).”
Orthogonal Estimation of Wasserstein Distances (2019, background): “Indeed, Hasselt (2010) observes that this phenomenon has a long history (see e.g. Smith and Winkler (2006); Harrison and March (1984)).”
An Intelligent Battery Controller Using Bias-Corrected Q-learning (2012, neutral): “The effect of this particular bias in applications has been observed earlier in estimating asset values in finance (Brown 1974), and by authors in reinforcement learning (Thurn and Schwartz 1993; Ormoneit and Sen 2002), management science (Smith and Winkler 2006) and in operations research (Powell 2”
Embedded Agency (2019, background): “It is more commonly known as the optimizer’s curse (Smith and Winkler 2006), and it is related to regression to the mean.”
Q-Learning: Theory and Applications (2020, methodology): “…considerably improved performance on the experience replay; Schaul et al. 2015); a method for reducing the finite-sample maximization bias (Smith & Winkler 2006) induced by the max operator (double Q-learning; Hasselt 2010, Hasselt et al. 2016); and the Noisy Net method for inducing and r…”
Optimization under Uncertainty in the Era of Big Data and Deep Learning: When Machine Learning Meets Mathematical Programming (2019, background): “Therefore, relying on a single probability distribution could result in suboptimal solutions, or even lead to the deterioration in out-of-sample performance [113].”
Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning (2019, background): “…that even if the distributional input parameters , implementing the optimal decisions leads to disappointment in out-of-sample tests. decision analysis this phenomenon is sometimes termed the optimizer’s curse [101], and stochastic optimization it is referred to as the optimization bias [27, 99].”
Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations (2015, background): “This phenomenon is termed the optimizer’s curse and is reminiscent of overfitting effects in statistics [48].”
Estimating the Maximum Expected Value in Continuous Reinforcement Learning Problems (2017, background): “(3) This estimator has a positive bias (Smith and Winkler 2006) that can be explained as follows.”
Double Q-learning (2010, result): “This result has been proven in previous work [16].”
Weighted Double Deep Multiagent Reinforcement Learning in Stochastic Cooperative Environments (2018, background): “However, Q-learning uses a single estimator to estimate E { max a (cid:48) Q ( s (cid:48) , a (cid:48) ) } , which has been proved to be greater than or equal to max a (cid:48) E { Q ( s (cid:48) , a (cid:48) ) } [ Smith and Winkler, 2006 ] .”
Estimating Maximum Expected Value through Gaussian Approximation (2016, background): “Unfortunately, as proved in (Smith & Winkler, 2006), this estimator is positively biased and this is critical in Q-Learning where the approximation error can increase step by step due to the overestimation of the state-action values.”