Autostacker: a compositional evolutionary learning system

By Harvey Wu, Warren Mo, Ishanu Chattopadhyay, Hod Lipson. 2018

87 citations recorded by the source corpus.

In this work, an automatic machine learning (AutoML) modeling architecture called Autostacker is introduced. Autostacker combines an innovative hierarchical stacking architecture and an evolutionary algorithm (EA) to perform efficient parameter search without the need for prior domain knowledge about the data or feature preprocessing. Using EA, Autostacker quickly evolves candidate pipelines with high predictive accuracy. These pipelines can be used in their given form, or serve as a starting point for further augmentation and refinement by human experts. Autostacker finds innovative machine learning model combinations and structures, rather than selecting a single model and optimizing its hyperparameters. When its performance on fifteen datasets is compared with that of other AutoML systems, Autostacker produces superior or competitive results in terms of both test accuracy and time cost.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Automated Machine Learning: State-of-The-Art and Open Challenges (2019, methodology): “Autostacker [23] is an AutoML framework that uses an evolutionary algorithm with hierarchical stacking for efficient hyper-parameters search.”
Benchmark and Survey of Automated Machine Learning Frameworks (2019, background): “Still, some features—like dates or addresses—can be transformed easily without domain knowledge to extract more meaningful features (Chen et al., 2018).”
Survey on Automated Machine Learning (2019, methodology): “Still, some features—like dates or addresses—can be easily parsed without domain knowledge to create more meaningful features (Chen et al., 2018).”
Auto-sklearn: Efficient and Robust Automated Machine Learning (2019, methodology): “Since the publication of the original NIPS paper [14] Auto-sklearn has become a standard baseline for new approaches to automated machine learning such as FLASH [39], RECIPE [32], Hyperband [25], Auto-Prognosis [2], ML-PLAN [28], Auto-Stacker [10] and AlphaD3M [11].”
Automatic Machine Learning by Pipeline Synthesis using Model-Based Reinforcement Learning and a Grammar (2019, unknown): “…such as diﬀerentiable programming (Milutinovic et al., 2017), tree search (Swearingen et al., 2017), evolutionary algorithm (Olson and Moore, 2016; Chen et al., 2018; Gijsbers et al., 2017), Bayesian optimization (Rasmussen, 2003; Bergstra and Bengio, 2012; Feurer et al., 2015; Kotthoﬀ et al.,…”
AlphaD3M: Machine Learning Pipeline Synthesis (2021, methodology): “, 2015), or on a small set of datasets (Chen et al., 2018), or on numerous datasets within specific domains (Olson and Moore, 2016).”
Techniques for Automated Machine Learning (2019, methodology): “Extant AutoMHL work with EA [25, 3, 27] focus on auto-”
Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning (2020, methodology): “Auto-Stacker went one step further, directly optimizing for a two-layer AutoML system (Chen et al., 2018).”
Scaling tree-based automated machine learning to biomedical big data with a feature set selector (2019, methodology): “For many bioinformatics problems of classifying individuals into clinical categories from high-dimensional biological data, performance of a machine learning (ML) model depends greatly on the problem it is applied to (Olson et al. , 2017, 2018).”