Design of the 2015 ChaLearn AutoML challenge

By Isabelle M Guyon, Kristin P. Bennett, Gavin C. Cawley, Hugo Jair Escalante, Sergio Escalera, Tin Kam Ho, Núria Macià, Bisakha Ray, Mehreen Saeed, Alexander R. Statnikov and 1 more. 2015

135 citations recorded by the source corpus.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Benchmark and Survey of Automated Machine Learning Frameworks (2019, methodology): “To solve some of these problems, the ChaLearn AutoML challenge (Guyon et al., 2015, 2016, 2018) has been introduced.”
Survey on Automated Machine Learning (2019, methodology): “To solve some of these problems, the ChaLearn AutoML challenge (Guyon et al., 2015, 2016, 2018) has been introduced.”
AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search (2019, background): “Hyper-parameter optimization is an important problem in Machine Learning [38, 14, 15], while Bayesian Optimization (BO) is a popular approach for the hyper-parameter search [17, 33, 38, 19].”
Automated Machine Learning: Methods, Systems, Challenges (2019, unknown): “One approach is to build a k-nearest neighbor (kNN) meta-model to predict which tasks are similar, and then rank the best configurations on these similar tasks [23, 147].”
Deep Learning in the Wild (2018, methodology): “Recent research in automated machine learning [17,40] and respective academic challenges [22] accurately aimed at ﬁnding a solution to this problem for sets of practically relevant use cases.”
Auto-sklearn: Efficient and Robust Automated Machine Learning (2019, methodology): “We note that balanced error or accuracy measures are often used in machine learning competitions, such as the AutoML challenge [17], which is described in Chapter 10.”
Efficient and Robust Automated Machine Learning (2015, unknown): “This problem definition reflects the setting of the ongoing ChaLearn AutoML challenge [1].”
Bayesian Hyperparameter Optimization for Ensemble Learning (2016, methodology): “(2015a) performed posthoc ensemble generation by reusing the product of a completed hyperparameter optimization, winning phase 1 of the ChaLearn AutoML challenge (Guyon et al., 2015).”
Towards Further Automation in AutoML (2018, methodology): “To the best of our knowledge, only Guyon et al. (2015) acknowledge the lack of such an approach; more specifically, they mention that “there is no attempt to treat the problem [of full model selection] as the optimization of a regularized functional J2 with respect to both (1) modeling choices and…”
Practical Automated Machine Learning for the AutoML Challenge 2018 (2018, background): “All performances are the normalized area under the ROC curve (Guyon et al., 2015).”
Probabilistic Matrix Factorization for Automated Machine Learning (2017, methodology): “In the rest of the paper, we will assume that Y is a matrix of balanced accuracies (see e.g., (Guyon et al., 2015)), and that we want to maximize the balanced accuracy for a new dataset; but, our approach can be used with any loss function ( e.g. RMSE, balanced error rate, etc.).”
Efﬁcient and Robust Automated Machine Learning (2015, unknown): “This problem definition reflects the setting of the ongoing ChaLearn AutoML challenge [1].”
A review of automatic selection methods for machine learning algorithms and hyper-parameter values (2016, background): “Automating machine learning model selection is a hot topic in computer science with an active open competition ongoing (Guyon et al. 2015).”
Bandit-Based Automated Machine Learning (2018, background): “Automated Machine Learning (AutoML) is the area that merges research efforts to automation of Machine Learning, which includes algorithm selection, model selection and hyperparameter optimization [3].”
A brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning without Human Intervention (2016, background): “The reader is referred to (Guyon et al., 2015) for a detailed description of the challenge.”
Towards Automatically-Tuned Neural Networks (2016, methodology): “With an early version of Auto-Net, we achieved the best performance on two datasets in the human expert track of the recent ChaLearn AutoML Challenge (Guyon et al., 2015).”