KeystoneML: Optimizing Pipelines for Large-Scale Advanced Analytics
By Evan R. Sparks, Shivaram Venkataraman, Tomer Kaftan, M. Franklin, B. Recht. 2016
161 citations recorded by the source corpus.
Modern advanced analytics applications make use of machine learning techniques and contain multiple steps of domain-specific and general-purpose processing with high resource requirements. We present KeystoneML, a system that captures and optimizes the end-to-end large-scale machine learning applications for high-throughput training in a distributed environment with a high-level API. This approach offers increased ease of use and higher performance over existing systems for large scale learning. We demonstrate the effectiveness of KeystoneML in achieving high quality statistical accuracy and scalable training using real world datasets in several domains.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sanjay Krishnan
- Eugene Wu
- Jan Gasthaus
- Tim Januschowski
- Steven Euijong Whang
- David Salinas
- Philipp Schmidt
- Sudip Roy
Citations in the corpus
- AlphaClean: Automatic Generation of Data Cleaning Pipelines (2019, methodology): “An initial architecture is to directly apply recent hyperparameter tuning approaches for machine learning pipelines and neural network model search [7, 20, 32, 33, 44].”
- Automating Large-Scale Data Quality Verification (2018, background): “…involved in building complex machine learning applications in the real-world have recently been highlighted by many researchers, e.g., with respect to managing the resulting models [26], software engineering [42, 8], pipeline abstractions [2, 43, 46], and learnings from real-world systems [34, 7].”
- TFX: A TensorFlow-Based Production-Scale Machine Learning Platform (2017, background): “Beyond simply stitching together components, a machine learning pipeline also needs to be simple to set up [16], maybe even support automated pipeline construction [20].”
- Data Lifecycle Challenges in Production Machine Learning (2018, background): “Feature engineering is a well-studied problem [7, 44, 43, 74, 58, 46, 8, 67] with a suite of techniques that are largely designed based on experience of ML experts.”
- Probabilistic Demand Forecasting at Scale (2017, background): “Many approaches [20, 27] build on variants of the popular pipeline abstraction in sci-kit learn [21].”