The Missing Piece in Complex Analytics: Low Latency, Scalable Model Management and Serving with Velox

By Daniel Crankshaw, Peter Bailis, Joseph E. Gonzalez, Haoyuan Li, Zhao Zhang, M. Franklin, A. Ghodsi, Michael I. Jordan. 2014

126 citations recorded by the source corpus.

To support complex data-intensive applications such as personalized recommendations, targeted advertising, and intelligent services, the data management community has focused heavily on the design of systems to support training complex models on large datasets. Unfortunately, the design of these systems largely ignores a critical component of the overall analytics process: the deployment and serving of models at scale. In this work, we present Velox, a new component of the Berkeley Data Analytics Stack. Velox is a data management system for facilitating the next steps in real-world, large-scale analytics pipelines: online model management, maintenance, and serving. Velox provides end-user applications and services with a low-latency, intuitive interface to models, transforming the raw statistical models currently trained using existing offline large-scale compute frameworks into full-blown, end-to-end data products capable of recommending products, targeting advertisements, and personalizing web content. To provide up-to-date results for these complex models, Velox also facilitates lightweight online model maintenance and selection (i.e., dynamic weighting). In this paper, we describe the challenges and architectural considerations required to achieve this functionality, including the abilities to span online and offline systems, to adaptively adjust model materialization strategies, and to exploit inherent statistical properties such as model error tolerance, all while operating at "Big Data" scale.

Subfields: Computer Networks and Communications, Information Systems and Management, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems and Management

In ML Systems & Hardware

Citations in the corpus

TensorFlow: A system for large-scale machine learning (2016, neutral): “Depending on the nature of the application, the inference may be required to produce results with very low latency in an interactive service, or execute on a disconnected mobile device.”
BoostClean: Automated Error Detection and Repair for Machine Learning (2017, methodology): “The database community has built systems to support almost every stage of the development process including featuriza-tion [1,53], distributed model training [2,17,19,24], and model deployment [16].”
Clipper: A Low-Latency Online Prediction Serving System (2016, neutral): “Velox is a UC Berkeley research project to study personalized prediction serving with Apache Spark.”
Big data analytics on Apache Spark (2016, background): “facilitating online model management, maintenance and serving [20].”
Ground: A Data Context Service (2017, background): “An interesting area of future research will be connecting prediction serving platforms (e.g., Velox [10] and Ten-sorFlow Serving [34]) to Ground.”
ProvDB: A System for Lifecycle Management of Collaborative Analysis Workflows (2016, background): “Several recent systems have attempted to address end-to-end issues, including model serving (e.g., TensorFlow Serving, Velox [9], MSMS [27], ModelDB [52]).”
Low latency RNN inference with cellular batching (2018, background): “LASER and Velox address issues such as how to re-train models quickly upon observing additional data during deployment, how to balance exploration vs. exploitation to gain useful feedback while maintaining good user experience.”
Making Contextual Decisions with Low Technical Debt (2016, background): “A few more, such as Google Cloud ML [27], Amazon ML [6], and AzureML [9] are designed to support development and deployment. However, these systems do not support data gathering or exploration. Velox [19] is an open-source system that supports model serving and batch training. Velox can adapt models”
Resource Central: Understanding and Predicting Workloads for Improved Resource Management in Large Cloud Platforms (2017, background): “Some prediction-serving systems [5, 6, 10] have been proposed recently, but have not been studied in the context of prediction-based resource management.”
TensorFlow-Serving: Flexible, High-Performance ML Serving (2017, methodology): “The general-purpose approaches we are aware of are Clipper [7], LASER [8] and Velox [9].”