TPOT: A Tree-based Pipeline Optimization Tool for Automating Machine Learning
By Randal S. Olson, Jason H. Moore. 2016
712 citations recorded by the source corpus.
As data science becomes increasingly mainstream, there will be an ever-growing demand for data science tools that are more accessible, flexible, and scalable. In response to this demand, automated machine learning (AutoML) researchers have begun building systems that automate the process of designing and optimizing machine learning pipelines. In this chapter we present TPOT v0.3, an open source genetic programming-based AutoML system that optimizes a series of feature preprocessors and machine learning models with the goal of maximizing classification accuracy on a supervised classification task. We benchmark TPOT on a series of 150 supervised classification tasks and find that it significantly outperforms a basic machine learning analysis in 21 of them, while experiencing minimal degradation in accuracy on 4 of the benchmarks—all without any domain knowledge nor human input. As such, genetic programming-based AutoML systems show considerable promise in the AutoML domain.
Subfields: Artificial Intelligence, Neural Architecture Search & AutoML
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Frank Hutter
- Marco F. Huber
- Marc-André Zöller
- Quoc V. Le
- Qiang Yang
- Yoshua Bengio
- Sergey Levine
- Mohit Bansal
In Neural Architecture Search & AutoML
- Frank Hutter
- Marc-André Zöller
- Marco F. Huber
- Marius Thomas Lindauer
- Li Yang
- Abdallah Shami
- Stefan Falkner
- Matthias Feurer
Citations in the corpus
- Towards modular and programmable architecture search (2019, background): “, for searching over Scikit-Learn pipelines [6]) and therefore lack support for general architecture search workflows.”
- Automated Machine Learning: State-of-The-Art and Open Challenges (2019, methodology): “The initial results of this approach has shown that the composable pipelines over Weka and Scikit-Learn does not significantly outperform the outcomes from Auto-Weka and Auto-Sklearn frameworks because it has to deal with larger search space.”
- Benchmark and Survey of Automated Machine Learning Frameworks (2019, methodology): “Regarding HPO, TPOT can only handle categorical parameters; similar to grid search all continuous hyperparameters have to be discretized.”
- Survey on Automated Machine Learning (2019, methodology): “Even though TPOT had the best overall performance, auto-sklearn is currently the best compromise for building complete ML pipelines automatically.”
- Random Search and Reproducibility for Neural Architecture Search (2019, background): “, evolutionary search [38, 19], Bayesian optimization [42, 4, 18], and gradient-based approaches [2, 34].”
- Automated Machine Learning: From Principles to Practices (2018, methodology): “However, this method only addresses the coupling between 𝑤 and 𝛼 but the coupling problem in 𝑤 still remains.”
- ATM: A distributed, collaborative, scalable system for automated machine learning (2017, methodology): “There are a few popular existing AutoML packages, such as Auto-WEKA [1], auto-sklearn [2], and TPOT [3].”
- Taking Human out of Learning Applications: A Survey on Automated Machine Learning (2018, methodology): “cess. The above methods have been widely applied in AutoML. For example, evolutionary algorithms has been applied in feature selection and generation [50], [86], [87], [88], [89], and model selection [90]. PSO has been used for model selection [53], [91], feature selection for support vector machine”
- Human-AI Collaboration in Data Science (2019, methodology): “Tools such as AutoWEKA [41, 66], Auto-sklearn [15], and TPOT [13, 52] all automate the model selection, hyperparameter optimization, and ensembling steps of the data science pipeline.”
- ML-Plan: Automated machine learning via hierarchical planning (2018, background): “2015), and TPOT (Olson and Moore 2016), which represent the state-of-the-art in AutoML.”
- Challenges in Deploying Machine Learning: A Survey of Case Studies (2020, methodology): “There is a large variety of tools that provide general-purpose implementations of AutoML algorithms, such as Auto-keras [137], Auto-sklearn [138] or TPOT [139].”
- SmartML: A Meta Learning-Based Framework for Automated Selection and Hyperparameter Tuning for Machine Learning Algorithms (2019, methodology): “Other tools include Google Vizier which is based on grid or random search [5] and TPOT which is based on genetic programming [10].”
- Automatic Machine Learning by Pipeline Synthesis using Model-Based Reinforcement Learning and a Grammar (2019, methodology): “…systems use methods such as differentiable programming (Milutinovic et al., 2017), tree search (Swearingen et al., 2017), evolutionary algorithm (Olson and Moore, 2016; Chen et al., 2018; Gijsbers et al., 2017), Bayesian optimization (Rasmussen, 2003; Bergstra and Bengio, 2012; Feurer et al.,…”
- Bayesian Optimization for Categorical and Category-Specific Continuous Inputs (2019, positive): “eropt-sklearn4 (using TPE for optimization) (Komer, Bergstra, and Eliasmith 2019), Auto-sklearn5 (using SMAC for optimization) (Feurer et al. 2019), and Tree-Based Pipeline Optimization Tool (TPOT6) (Olson and Moore 2019). All methods are applied to the same training and test sets and repeated 10 ti”
- Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence (2020, background): “Currently, Auto-sklearn [49], TPOT [50], H2O-AutoML [51], Microsoft’s NNI 11, and AutoKeras [52] are the most popular ones among practitioners and further discussed in this section.”
- AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data (2020, methodology): “The Tree-based Pipeline Optimization Tool (TPOT) of Olson & Moore (2019) employs genetic algorithms to optimize ML pipelines.”
- AVATAR - Machine Learning Pipeline Evaluation Using Surrogate Model (2020, methodology): “es during the composition and optimisation, these approaches limit the exploration of promising pipelines which may have a variety of structures. Secondly, while evolutionary algorithms based methods [4] enable the randomness of the pipelines’ structure using the concept of evolution, this randomnes”
- On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice (2020, methodology): “The involved ML and HPO algorithms are evaluated using multiple open-source Python libraries and frameworks introduced in Section 6, including sklearn [31], Skopt [114], Hyperopt [110], Optunity [83], Hyperband [16], BOHB [97], and TPOT [122].”