Automated configuration of algorithms for solving hard computational problems

By Frank Hutter. 2009

119 citations recorded by the source corpus.

Subfields: Control and Systems Engineering, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

In Neural Architecture Search & AutoML

Citations in the corpus

Hyper-Parameter Optimization: A Review of Algorithms and Applications (2020, background): “For vanilla Bayesian, kernel function and optimization process must be adapted to involve integers (e.g., batch size) and categorical data (Hutter, 2009; GarridoMerchán and Hernández-Lobato, 2017).”
Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces (2014, background): “This is a general problem in algorithm conﬁguration [6] that occurs in many machine learning contexts, such as, e.g., in deep neural networks [7]; ﬂexible computer vision architectures [8]; and the combined selection and hyperparameter optimization of machine learning algorithms [9].”
Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, background): “Since EGO’s publication, there has evolved a body of work devoted to extending the algorithm, particularly in adding constraints to the optimization problem [6], [131], [23], and in modelling noisy functions [14], [75], [76].”
Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages (2015, background): “for random forests remains a challenging topic of current research (Hutter, 2009).”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, background): “There are some interesting steps in this direction (Hutter, 2009; Bergstra et al. , 2011; Hutter et al. , 2011; Srinivasan and Ramakrishnan, 2011) but much more needs to done.”
ParamILS: An Automatic Algorithm Configuration Framework (2014, methodology): “(For full details, see Section 8.1 of Hutter, 2009).”
A Survey of Automatic Parameter Tuning Methods for Metaheuristics (2020, unknown): “[28], [29] stated”
UvA-DARE (Digital Academic Repository) Bayesian Optimization in High Dimensions via Random Embeddings (2013, unknown): “In our experiments, following [ Hutter, 2009 ] , we deﬁned g ( x (1) , x (2) ) =”
Scalable Nearest Neighbor Algorithms for High Dimensional Data (2014, background): “There has been extensive research on algorithm configuration methods [44], [45], however we are not aware of papers that apply such techniques to finding optimum parameters”
Active Learning of Linear Embeddings for Gaussian Processes (2013, methodology): “Within Bayesian optimization, much recent work has focused on high-dimensional problems (Hut-ter et al., 2011; Chen et al., 2012; Carpentier & Munos, 2012; Bergstra & Bengio, 2012; Hutter, 2009).”
Efficient global optimization for combinatorial problems (2014, methodology): “Another approach to a mixed problem is taken by Hutter [12], who describes a Kriging model based on a weighted Hamming distance to model categorical variables for algorithm tuning.”
AutoHAS: Efficient Hyperparameter and Architecture Search. (2020, unknown): “Black-box and multi-ﬁdelity HPO methods have a long standing history (Bergstra & Bengio, 2012; Hutter, 2009; Hutter et al., 2011; 2019; Kohavi & John, 1995; Hutter et al., 2019).”
Hydra-MIP : Automated Algorithm Configuration and Selection for Mixed Integer Programming (2011, methodology): “We constructed a large set of 139 MIP features, drawing on 97 existing features [21, 11, 17] and also including 42 new probing features.”
Automated Algorithm Configuration and Parameter Tuning (2012, methodology): “Finally, work originating from the Ph.D. project of Hutter (2009) has recently overcome two major limitations of the sequential model-based optimisation methods discussed in Section 4 of this chapter by introducing a procedure that can handle categorical parameters while explicitly exploiting the…”
Sample-Efficient Neural Architecture Search by Learning Action Space (2019, methodology): “Sequential Model Based Optimizations (SMBO) is a classic black box optimization framework [6], [46], that uses a surrogate S to extrapolate unseen region in Ω and to interpolate the explored region with existing samples.”
Algorithms for Hyper-Parameter Optimization (2011, methodology): “Sequential Model-Based Global Optimization (SMBO) algorithms have been used in many applications where evaluation of the fitness function is expensive [8, 9].”
Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures (2013, methodology): “Choice nodes make it possible to encode conditional parameters in a search space (Hutter, 2009).”
Automated Configuration of Mixed Integer Programming Solvers (2010, methodology): “Partly to address those issues, we are actively developing an alternative algorithm configuration approach that is based on response surface models [17, 18, 15].”
Identifying Key Algorithm Parameters and Instance Features Using Forward Selection (2013, methodology): “Specifically, it has been convincingly demonstrated that methods for automated algorithm configuration [2, 3, 4, 5, 6, 7] are able to find configurations that substantially improve the state of the art for various hard combinatorial problems (e.”