On Bayesian Methods for Seeking the Extremum

By Jonas Mockus. 1974

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Hyper-Parameter Optimization: A Review of Algorithms and Applications (2020, background): “Matérn kernel with an automatic relevance determination (Rasmussen, 2003) and expected improvement acquisition function (Mockus et al., 1978) are applied.”
Bayesian Neural Architecture Search using A Training-Free Performance Metric (2020, methodology): “Efficient Global ptimization) [8,9] algorithm has been applied extensively for utomated algorithm configuration tasks [25–27].”
BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search (2019, methodology): “The next architecture is then chosen by optimizing an acquisition function such as expected improvement (EI) (Močkus 1975) or Thompson sampling (TS) (Thompson 1933).”
Learning to select data for transfer learning with Bayesian Optimization (2017, methodology): “We use GP with Monte Carlo acquistion and Expected Improvement (EI) (Moˇckus, 1974) as acquisition function as this combination has been shown to outperform comparable approaches (Snoek et al., 2012).”
High-Dimensional Gaussian Process Bandits (2013, background): “The use of GPs to negotiate exploration– exploitation tradeoffs originated in the areas of response surface and Bayesian optimization, for which there are a number of approaches (cf., [13]), perhaps most notably the Expected Improvement [14] approach, which has recently received theoretical justific”
ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language (2019, background): “|D n ; There are a number of popular acquisition functions used commonly in Bayesian optimization, such as expected improvement (EI) [34], probability of improvement (PI) [26], GP upper conﬁdence bound (UCB) [46], and Thompson sampling (TS) [48].”
Artificial intelligence in medicine (1989, unknown): “Examples of these works include stacked auto-encoders [15,16], grid search, random search [17], Bayesianbased Gaussian process (BGP) [18,19], tree-structured Parzen estimators (TPE) [20], Reinforcement learning (RL) [21] and sequential model-based global optimization (SMBO) [22].”
Automatic Discovery of Privacy–Utility Pareto Fronts (2019, background): “In the single-objective case, a common choice is to select the location that, in expectation under the model, gives the best improvement to the current estimate [40].”
GFlowNet Foundations (2021, unknown): “We can use the acquisition function heuristics (like Upper Confidence Bound (UCB) or Expected Improvement (EI)) from Bayesian optimization (Močkus, 1975; Srinivas et al., 2010) to combine the predicted usefulness f̂(x) of configuration x with an estimate of the epistemic uncertainty around that pred”
Dealing with Categorical and Integer-valued Variables in Bayesian Optimization with Gaussian Processes (2017, background): “Bayesian Optimization (BO) methods are popular for optimizing black-box functions with the characteristics described (Moˇckus, 1975).”
Cooling Strategies for the Moment-Generating Function in Bayesian Global Optimization (2018, methodology): “2) are generated from the Latin hypercube sampling with design ranges: t0 ∈ [1,5] and tf ∈ [10−3,0.”
Model-based methods for continuous and discrete global optimization (2017, methodology): “A popular adaptive sampling method is expected improvement (EI) [29, 30], which is discussed in Jones’ seminal paper [31].”
DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering (2020, methodology): “For each of the upper layers k + 1 through n, we compute a layerwise representation similarity (lrs) loss as follows: Llrs = n∑ i=k m∑ j=1 ‖vij − uij‖2 We add the knowledge distillation loss (Lkd) and layerwise representation similarity loss (Llrs) along with the task specific supervision Loss (Lts)”
Completely Automated CNN Architecture Design Based on Blocks (2020, methodology): “…optimization algorithms based on the heuristic computational paradigms [9], such as random search [10], Bayesian-based Gaussian process [11], [12], tree-structured Parzen estimators [13], sequential model-based global optimization [14], neuroevolution of augmenting topologies [15], and…”
PipeDream: Fast and Efficient Pipeline Parallel DNN Training (2018, methodology): “Guided by prior work, we adjust the learning rate during training to converge to the desired result faster [31, 37, 38, 13, 24].”
A new acquisition function for Bayesian optimization based on the moment-generating function (2017, methodology): “This optimization framework is proposed by Jonas Mockus and Antanas Zilinskas, et. al.”