TuPAQ: An Efficient Planner for Large-scale Predictive Analytic Queries

By Evan R. Sparks, Ameet Talwalkar, M. Franklin, Michael I. Jordan, Tim Kraska. 2015

26 citations recorded by the source corpus.

The proliferation of massive datasets combined with the development of sophisticated analytical techniques have enabled a wide variety of novel applications such as improved product recommendations, automatic image tagging, and improved speech-driven interfaces. These and many other applications can be supported by Predictive Analytic Queries (PAQs). A major obstacle to supporting PAQs is the challenging and expensive process of identifying and training an appropriate predictive model. Recent efforts aiming to automate this process have focused on single node implementations and have assumed that model training itself is a black box, thus limiting the effectiveness of such approaches on large-scale problems. In this work, we build upon these recent efforts and propose an integrated PAQ planning architecture that combines advanced model search techniques, bandit resource allocation via runtime algorithm introspection, and physical optimization via batching. The result is TuPAQ, a component of the MLbase system, which solves the PAQ planning problem with comparable quality to exhaustive strategies but an order of magnitude more efficiently than the standard baseline approach, and can scale to models trained on terabytes of data across hundreds of machines.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Tong Yu
2. Yimin Huang
3. Hong Zhu
4. Kevin G. Jamieson
5. Yujun Li
6. Yeounoh Chung
7. Zhenguo Li
8. Salman Salloum

Citations in the corpus

• Hyper-Parameter Optimization: A Review of Algorithms and Applications (2020, methodology): “The SHA model was built upon the multi-armed bandit formulation (Agarwal et al., 2012; Sparks et al., 2015).”
• Non-stochastic Best Arm Identification and Hyperparameter Optimization (2015, methodology): “We build upon these previous works, and in particular study the multi-armed bandit formulation proposed in [7] and [9], where each arm corresponds to a fixed hyperparameter setting, pulling an arm corresponds to a fixed”
• Big data analytics on Apache Spark (2016, background): “A new component calledTuPAQ (Training-supported Predictive Analytic Query Planner) [74,75] automatically finds and trains models for large-scale machine learning.”
• An Asymptotically Optimal Multi-Armed Bandit Algorithm and Hyperparameter Optimization (2020, methodology): “In this section, we prove that the proposed SS method is asymptotically optimal using the tools from multi-armed bandit (MAB) (Agarwal et al., 2012; Sparks et al., 2015; Jamieson and Talwalkar, 2016).”
• Democratizing Data Science through Interactive Curation of ML Pipelines (2019, background): “Spark TuPAQ [36] and Hyperband [22] use variations of multi-armed bandit (MAB) algorithm to better allocate computational resources for hyper-parameter tuning.”