Parallel Algorithm Configuration

By Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown. 2012

95 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Hardware and Architecture

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Hardware and Architecture

Citations in the corpus

Hyper-Parameter Optimization: A Review of Algorithms and Applications (2020, methodology): “Various acquisition functions have been proposed in previous studies, including the probability of improvement (Kushner, 1964), GP upper confidence bound (GP-UCB) (Lai and Robbins, 1985; Srinivas et al., 2009), predictive entropy search (Hernández-Lobato et al., 2014), and even a portfolio containi”
Benchmark and Survey of Automated Machine Learning Frameworks (2019, background): “Executing multiple SMBO instances with different random seeds allows a simple parallelization (Hutter et al., 2012).”
Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, methodology): “[80] proposed an approach based on GP-UCB where UCB is optimized using a range of n values, which produces a set of points that favor a range of exploration and exploitation.”
CMA-ES for Hyperparameter Optimization of Deep Neural Networks (2016, neutral): “, 2012a) or by evaluating the optima of multiple acquisition functions concurrently (Hutter et al., 2012), perfect parallelization appears unattainable since the decisions in each step depend on all data points gathered so far.”
Algorithm Selection for Combinatorial Search Problems: A Survey (2012, background): “This approach is used for example in SATzilla (Nudelman et al., 2004; Xu, Hutter, Hoos, & Leyton-Brown, 2007; Xu et al., 2008), AQME (Pulina & Tacchella, 2007, 2009), CPhy-dra (O’Mahony, Hebrard, Holland, Nugent, & O’Sullivan, 2008), ArgoSmArT (Nikoli´c, Mari´c, & Janiˇci´c, 2009) and BUS (Howe, Dah”
A new acquisition function for Bayesian optimization based on the moment-generating function (2017, background): “[31], instantiates multiple LCB functions by sampling several β values from an exponential distribution with the unit mean.”
Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges (2021, background): “For this so-called qLCB (also referred to as qUCB) approach, Hutter et al. (2012) propose to draw κ from an exponential distribution with rate parameter 1.”
Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms (2012, methodology): “3Other, more sophisticated methods for the parallelization of Bayesian optimization exist [17, 4, 11, 29], but to date, there is no empirical evidence that these methods outperform the simple approach we use here when the cost of evaluating hyperparameter configurations varies across the space.”
UvA-DARE (Digital Academic Repository) Bayesian Optimization in High Dimensions via Random Embeddings (2013, unknown): “Figure 4 (top) compares BO, REMBO, and the baseline random search against ParamILS (which was used for all con-ﬁguration experiments in [Hutter et al. , 2010]) and SMAC [Hutter et al. , 2011].”
Bayesian Optimization of Text Representations (2015, background): “There has been work to parallelize Bayesian optimization, making it possible to leverage the power of multicore architectures (Snoek et al., 2012; Desautels et al., 2012; Hutter et al., 2012).”
Phoenics: A Bayesian Optimizer for Chemistry (2018, methodology): “− 35 Chemical systems can have complex, qualitatively di ﬀ erent response surfaces.”
Bayesian Optimization in a Billion Dimensions via Random Embeddings (2013, methodology): “However, as shown by Hutter et al. (2012), multiple independent runs can also improve the performance of SMAC and especially ParamILS.”
Algorithm runtime prediction: Methods & evaluation (2012, methodology): “First, they can model the performance of a parameterized algorithm dependent on the settings of its parameters; in a sequential model-based optimization process, one alternates between learning an EPM and using it to identify promising settings to evaluate next [65,7,59,55,56].”
Auto-WEKA : combined selection and hyperparameter optimization of supervised machine learning algorithms (2014, methodology): “Other, more sophisticated methods for the parallelization of Bayesian optimization exist [Hutter et al., 2012, Bergstra et al., 2011, Desautels et al., 2012, Snoek et al., 2012], but to date, there is no empirical evidence that these methods outperform the simple approach we used here when the cost…”