Model Selection: Beyond the Bayesian/Frequentist Divide

By Isabelle M Guyon, Amir Saffari, Gideon Dror, Gavin C. Cawley. 2010

183 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

Automated Machine Learning: Methods, Systems, Challenges (2019, unknown): “While there exists much work on parallel Bayesian optimization [12, 24, 33, 44, 54, 60, 135, 140], except for the neural networks described in Sect.”
Tikhonov, Ivanov and Morozov regularization for support vector machine learning (2016, methodology): “Consequently, according to the SRM principle, the control of the hypothesis space size assumes a central role in learning (Guyon et al. 2010; Anguita et al. 2011a).”
Sparse Signal Recovery With Temporally Correlated Source Vectors Using Sparse Bayesian Learning (2011, background): “Before estimating the hyperparameters, we note that assigning a different matrixBi to each sourceXi· will result in overfitting [46], [47] due to limited data and too many parameters.”
The 'K' in K-fold Cross Validation (2012, methodology): “Several techniques can be exploited for MS and EE in Support Vector Machines [2, 3].”
Feature selection via regularized trees (2012, methodology): “Feature selection methods can be divided into filters, wrappers and embedded methods [5].”
Machine learning approaches for improving condition-based maintenance of naval propulsion plants (2016, background): “The most widespread quality index for ML models is the estimated generalization error of the regressor, that is, the error that the function will be likely performing on new unknown data, generated by the same probability distribution that originated the learning set.(61) Several methods exist for t”
Efficient and Robust Automated Machine Learning (2015, unknown): “It is well known that ensembles often outperform individual models [22, 23], and that effective ensembles can be created from a library of models [24, 25].”
What is an optimal value of k in k-fold cross-validation in discrete Bayesian network analysis? (2020, methodology): “In our model parameterization and testing, we used a sequential optimization procedure whereby key parameters were fixed and other parameters were then optimized iteratively (some call this a multilevel inference problem, e.g., Guyon et al. 2010).”
Efﬁcient and Robust Automated Machine Learning (2015, unknown): “It is well known that ensembles often outperform individual models [22, 23], and that effective ensembles can be created from a library of models [24, 25].”
Tunability: Importance of Hyperparameters of Machine Learning Algorithms (2018, methodology): “Alternatively, one can use hyperparameter tuning strategies , which are data-dependent, second-level optimization procedures (Guyon et al., 2010), which try to minimize the expected generalization error of the inducing algorithm over a hyperparameter search space of considered candidate…”
Open challenges for data stream mining research (2014, methodology): “Recently, model selection methods have been proposed that do not require internal crossvalidation, but rather use the Bayesian machinery to design regularizers with data dependent priors [20].”
Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges (2021, background): “There are mainly two reasons why such a bilevel optimization is preferable to a direct, joint risk minimization of parameters and HPs (Guyon et al., 2010): (cid:129) Typically, learners are constructed in such a way that optimized first-level parameters can be more efficiently computed for fixed…”
On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation (2010, background): “…are based on some form of multi-level inference, where the model is deﬁned by a set of model parameters and also a set of hyper-parameters (Guyon et al., 2009), for example in kernel learning methods the parameters correspond to the coefﬁcients of the kernel expansion and the…”
Auto-sklearn: Efficient and Robust Automated Machine Learning (2019, background): “1 It is well known that ensembles often outperform individual models [18, 24], and that eﬀective ensembles can be created from a library of models [9, 8].”