Efﬁcient and Robust Automated Machine Learning

By Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Tobias Springenberg, Manuel Blum, Frank Hutter. 2015

Subfields: Efficiency & Compression, Neural Architecture Search & AutoML

Citations in the corpus

AutoML: A Survey of the State-of-the-Art (2019, neutral): “Although there are several pipeline libraries proposed, such as TPOT [138] and Auto-Sklearn [139], they all lack the procedure of data collection, a process that is usually ﬁnished manually and therefore time-consuming and tedious.”
A System for Massively Parallel Hyperparameter Tuning (2018, neutral): “AutoWEKA (Kotthoff et al., 2017) and AutoSklearn (Feurer et al., 2015) are two established single-machine, single-user systems for hyperparameter optimization.”
Deep Neural Architecture Search with Deep Graph Bayesian Optimization (2019, unknown): “BO has a large number of applications including hyperparameter tuning for machine learning models [ Feurer et al. , 2015; Thornton et al. , 2013 ] .”
Regularized Evolution for Image Classifier Architecture Search (2018, unknown): “Several recent papers reduced the compute cost through progressive-complexity search stages [28], hyper-nets [5], accuracy prediction [3, 13, 25], warm-starting and ensembling [17], parallelization, reward shaping and early stopping [51] or Net2Net transformations [6].”
One button machine for automating feature engineering in relational databases (2017, unknown): “Auto-Weka [8], [11] and Auto-SkLearn [3] are among the ﬁrst works trying to ﬁnd the best combination of data preprocessing, hyper-parameter tuning and model selection.”
AutoPrognosis: Automated Clinical Prognostic Modeling via Bayesian Optimization with Structured Kernel Learning (2018, unknown): “Existing methods for meta-learning focus only on identifying well-performing pipelines from other datasets, and use them for initializing the optimization procedure (Brazdil et al., 2008; Feurer et al., 2015).”
Hyperparameter Ensembles for Robustness and Uncertainty Quantification (2020, unknown): “Given the simplicity, broad applicability, and performance of the greedy algorithm from [11]—e.g., in auto-ML settings [18], we use it as our canonical procedure to generate a “row”, i.e., an ensemble of neural networks with ﬁxed parameter initialization and various hyperparameters.”
Warmstarting of Model-based Algorithm Configuration (2017, unknown): “…the propositional satisﬁability problem SAT (Hutter et al. 2017) or AI planning (Fawcett et al. 2011)) as well as a wide range of machine learning algorithms (in particular deep neural networks (Snoek, Larochelle, and Adams 2012) and automated machine learning frameworks (Feurer et al. 2015)).”
MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis (2020, unknown): “We have compared the performance of several baseline methods, including early-stopping ResNets [6], open-source AutoML tools (auto-sklearn [7] and AutoKeras [8]) and a commercial AutoML tool (Google AutoML Vision).”
Learning Feature Engineering for Classification (2017, unknown): “To characterize datasets, hand-crafted meta-features, ﬁxed-size stratiﬁed sampling, neural networks and hashing methods have been used for different tasks [Michie et al. , 1994; Kalousis, 2002; Feurer et al. , 2015; Weinberger et al. , 2009].”
Transferable AutoML by Model Sharing Over Grouped Datasets (2019, unknown): “Splitting dataset into sub-dataset is widely used in multi-task learning and transfer learning [26, 35].”
AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data (2020, unknown): “After models have been selected, Auto-WEKA tries various ensembling strategies to further improve its predictions. auto-sklearn (Feurer et al., 2015) has been the winner of numerous AutoML competitions to date (Feurer et al., 2018; 2019; Guyon et al., 2019).”
LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks (2022, unknown): “Furthermore, our work shares the general goal with automated machine learning (AutoML) [90, 91] in improving the usability of machine learning, though LIFT uses only a single pretrained LMs for all tasks while AutoML automates the standard machine learning pipeline from a set of existing algorithms.”
TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second (2022, unknown): “…10 000 configurations were eval-7 Erickson et al., 2020), which combines ML models including neural networks and tree-based models into a stacked ensemble, and Auto-sklearn 2.0 (Feurer et al., 2015; 2021) which uses Bayesian Optimization and combines the evaluated models into a weighted ensemble.”
Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach (2016, unknown): “1 We use the term robustness in its meaning of immunity to noise and not generically as “adaptivity to various scenarios”, e.g. [6].”
Automated Machine Learning in Practice: State of the Art and Recent Results (2019, unknown): “Auto-sklearn [17]: We used the algorithm explained in Section III to ﬁnd the best pipeline within the time-budget computed for training 100 models by DSM. Auto-sklearn is slow in start [17]; however, it eventually reaches a solution very close to the optimum.”