A review of automatic selection methods for machine learning algorithms and hyper-parameter values

By Gang Luo. 2016

340 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Marco F. Huber
2. Marc-André Zöller
3. Philipp Probst
4. Anne-Laure Boulesteix
5. Li Yang
6. Abdallah Shami
7. Yeounoh Chung
8. Bernd Bischl

In Molecular Biology

1. Philipp Probst
2. Bernd Bischl
3. Marc-André Zöller
4. Hussain Alibrahim
5. Simone A. Ludwig
6. Li Yang
7. Abdallah Shami
8. Anne-Laure Boulesteix

Citations in the corpus

• Benchmark and Survey of Automated Machine Learning Frameworks (2019, methodology): “Consequently, efficient solvers, like convex or gradient-based optimization, cannot be used for Equation (2) (Luo, 2016).”
• Survey on Automated Machine Learning (2019, methodology): “Consequently, efficient solvers, like convex or gradient-based optimization, cannot be used for Equation (1) (Luo, 2016).”
• Meta-Learning: A Survey (2018, neutral): “There exists a rich body of earlier work [22, 56, 87, 94] on building meta-models for algorithm selection [15, 19, 70, 115] and hyperparameter recommendation [4,”
• Machine Learning Applications in Epilepsy (2019, background): “16 With small datasets or excessively complex models, the mapping function may “overfit” the training data, learning properties inherent to the sample rather than relationships in the overall population; as a result, the generalizability, or out‐of‐sample accuracy, of the algorithm is diminished.”
• Hyperparameter Optimization: Comparing Genetic Algorithm against Grid Search and Bayesian Optimization (2021, methodology): “A machine learning model has two types of parameters [6]:”
• Tunability: Importance of Hyperparameters of Machine Learning Algorithms (2018, background): “For a recent overview of tuning strategies, see, e.g., Luo (2016).”
• On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice (2020, methodology): “However, most traditional optimization techniques [25] are unsuitable for HPO, since HPO problems are different from traditional optimization problems in the following aspects [10]:”
• Democratizing Data Science through Interactive Curation of ML Pipelines (2019, background): “AutoML Systems: Most automated ML systems focus on automated learning algorithm selection and hyper-parameter tuning [4, 6, 13, 23, 24, 36, 45] to make machine learning curation fully automated for non-ML experts.”