Auto-sklearn: Efficient and Robust Automated Machine Learning

By Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Tobias Springenberg, Manuel Blum, Frank Hutter. Automated Machine Learning, 2019

354 citations recorded by the source corpus.

The success of machine learning in a broad range of applications has led to an ever-growing demand for machine learning systems that can be used off the shelf by non-experts. To be effective in practice, such systems need to automatically choose a good algorithm and feature preprocessing steps for a new dataset at hand, and also set their respective hyperparameters. Recent work has started to tackle this automated machine learning (AutoML) problem with the help of efficient Bayesian optimization methods. Building on this, we introduce a robust new AutoML system based on the Python machine learning package scikit-learn (using 15 classifiers, 14 feature preprocessing methods, and 4 data preprocessing methods, giving rise to a structured hypothesis space with 110 hyperparameters). This system, which we dub Auto-sklearn, improves on existing AutoML methods by automatically taking into account past performance on similar datasets, and by constructing ensembles from the models evaluated during the optimization. Our system won six out of ten phases of the first ChaLearn AutoML challenge, and our comprehensive analysis on over 100 diverse datasets shows that it substantially outperforms the previous state of the art in AutoML. We also demonstrate the performance gains due to each of our contributions and derive insights into the effectiveness of the individual components of Auto-sklearn.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/54b2c43d93d1f525213da5ddc70db5d27c1120f4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Bayesian Optimization for Categorical and Category-Specific Continuous Inputs (2019, neutral): “For example, consider the problem of automated machine learning (Feurer et al. 2019) where we need to automatically select the best performing machine learning model along with its optimal hyper-parameters.”
Automatic tuning of hyperparameters using Bayesian optimization (2020, methodology): “Bayesian optimization is also widely used in diverse design problems in different fields, such as gait recognition (Feurer et al. 2019), environmental sustainment and monitoring (Marchant and Ramos 2012), algorithm configuration using optimization (Hutter et al. 2014), automatic machine learning…”
Toxic Language Detection in Social Media for Brazilian Portuguese: New Dataset and Multilingual Analysis (2020, methodology): “For this, we use the auto-sklearn8 library (Feurer et al., 2019).”
Amazon SageMaker Autopilot: a white box AutoML solution at scale (2020, background): “Auto-sklearn (Feurer et al., 2019) has dominated a number of AutoML competitions.”
MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification (2021, neutral): “30 Abdominal CT MC (11) 25,221 13,940 / 2,452 / 8,829”
Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence (2020, background): “Auto-sklearn makes use of meta-learning, allowing it to learn from previously trained datasets while both Auto-sklearn and H2O-AutoML provide options to avoid parameters that are known to cause slow optimization.”
Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation (2020, methodology): “On average, the Selected individual model under standard training (BASE) would be outperformed by these AutoML ensembles, but surprisingly, our distillation approach produces Selected individual models that are both more accurate and over 10 ˆ more efﬁcient than H2O and AutoSklearn.”
AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data (2020, methodology): “After models have been selected, Auto-WEKA tries various ensembling strategies to further improve its predictions. auto-sklearn (Feurer et al., 2015) has been the winner of numerous AutoML competitions to date (Feurer et al., 2018; 2019; Guyon et al., 2019).”