Autmatic parameter selection by minimizing estimated error

By Ron Kohavi, George H. John. 1995

Subfields: Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Neural Architecture Search & AutoML

Citations in the corpus

Advances and Open Problems in Federated Learning (2019, background): “Hyperparameter optimization (HPO) has a long history under the framework of AutoML [339, 237, 241], but it mainly concerns how to improve the model accuracy [59, 364, 321, 159] rather than communication and computing efficacy for”
AutoHAS: Differentiable Hyper-parameter and Architecture Search (2020, methodology): “on to decide which candidate to evaluate next [49]. Multi-ﬁdelity optimization methods accelerate the above methods by evaluating on a proxy task, e.g., using less training epochs or a subset of data [9, 23, 27, 31]. These HPO methods are computationally expensive to search for deep learning models ”
Hyper-Parameter Optimization: A Review of Algorithms and Applications (2020, methodology): “tensions This section concludes the paper with a discussion on the comparison between algorithms (Section 5.1) and evaluations of HPO (Section 5.2). The concept of HPO has a history of over 20 years (Kohavi and John, 1995), and it is a widely used mature technique. In the last few years, with the pr”
Benchmark and Survey of Automated Machine Learning Frameworks (2019, methodology): “Adaptations of grid search to test possible configurations in a greedy best-first approach are available since 1995 (Kohavi & John, 1995).”
Meta-Learning: A Survey (2018, neutral): “Learning curves are used extensively to speed up hyperparameter optimization on a given task (Kohavi and John, 1995; Provost et al., 1999; Swersky et al., 2014; Chandrashekaran and Lane, 2017).”
AutoHAS: Efficient Hyperparameter and Architecture Search. (2020, methodology): “Black-box and multi-ﬁdelity HPO methods have a long standing history (Bergstra & Bengio, 2012; Hutter, 2009; Hutter et al., 2011; 2019; Kohavi & John, 1995; Hutter et al., 2019).”
Automated configuration of algorithms for solving hard computational problems (2009, background): “…(Cavazos and O’Boyle, 2005), parallel computing (Brewer, 1995), computer vision (Muja and Lowe, 2009), machine learning (Maron and Moore, 1994; Kohavi and John, 1995), database query optimization (Stillger and Spiliopoulou, 1996), database server optimization (Diao et al., 2003), protein…”
A Meta-Learning Method to Select the Kernel Width in Support Vector Regression (2004, result): “We plan to investigate some of these cases in future and compare our approach to other general parameter setting methods (Kohavi & John, 1995).”
Decision tree classifier for network intrusion detection with GA-based feature selection (2005, methodology): “Our proposed GA-based Feature Selection algorithm is based on the wrapper model [12][13] as discussed in section 2.”
Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning (2020, background): “…2015; Tsamardinos et al., 2018), but while such model selection strategies are an important area of research, houldout or K-fold cross-validation remain the most prominent choices (Henery, 1994; Kohavi and John, 1995; Hastie et al., 2001; Guyon et al., 2010; Bischl et al., 2012; Raschka, 2018).”
Fast cross-validation via sequential testing (2012, methodology): “For example, instead of using the full grid, local search heuristics may be used to find local minima in the test error (see for instance Kohavi and John 1995; Bengio 2000; Keerthi et al. 2006).”
Multi-Objective Parameter Selection for Classifiers (2012, background): “Kohavi and John (1995) introduce a general framework that optimizes a classiﬁer according to its cross-validation error in a best-ﬁrst search algorithm.”
Separate-and-Conquer Rule Learning (1999, background): “Kohavi and John (1995) describe an approach how such parameters could be automatically adjusted.”