Stale View Cleaning: Getting Fresh Answers from Stale Materialized Views

By Sanjay Krishnan, Jiannan Wang, M. Franklin, Ken Goldberg, Tim Kraska. 2015

28 citations recorded by the source corpus.

Materialized views (MVs), stored pre-computed results, are widely used to facilitate fast queries on large datasets. When new records arrive at a high rate, it is infeasible to continuously update (maintain) MVs and a common solution is to defer maintenance by batching updates together. Between batches the MVs become increasingly stale with incorrect, missing, and superfluous rows leading to increasingly inaccurate query results. We propose Stale View Cleaning (SVC) which addresses this problem from a data cleaning perspective. In SVC, we efficiently clean a sample of rows from a stale MV, and use the clean sample to estimate aggregate query results. While approximate, the estimated query results reflect the most recent data. As sampling can be sensitive to long-tailed distributions, we further explore an outlier indexing technique to give increased accuracy when the data distributions are skewed. SVC complements existing deferred maintenance approaches by giving accurate and bounded query answers between maintenance. We evaluate our method on a generated dataset from the TPC-D benchmark and a real video distribution application. Experiments confirm our theoretical results: (1) cleaning an MV sample is more efficient than full view maintenance, (2) the estimated results are more accurate than using the stale MV, and (3) SVC is applicable for a wide variety of MVs.

Subfields: Computer Networks and Communications, Management Science and Operations Research, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Sanjay Krishnan
2. Xu Chu
3. Ihab F. Ilyas
4. Eugene Wu
5. Ken Goldberg
6. Jiannan Wang
7. M. Franklin
8. Tova Milo

In Management Science and Operations Research

1. Sanjay Krishnan
2. Eugene Wu
3. M. Franklin
4. Jiannan Wang
5. Ihab F. Ilyas
6. Ken Goldberg
7. Xu Chu
8. Tova Milo

In Signal Processing

1. Sanjay Krishnan
2. Ihab F. Ilyas
3. Eugene Wu
4. Jiannan Wang
5. M. Franklin
6. Ken Goldberg
7. Xu Chu
8. Tim Kraska

Citations in the corpus

• ActiveClean: An Interactive Data Cleaning Framework For Modern Machine Learning (2016, methodology): “The selection technique applied in ActiveClean uses pointwise gradients to generalize the outlier filtering heuristics to select potentially dirty data even in complex models.”
• Wisteria: Nurturing Scalable Data Cleaning Infrastructure (2015, background): “In prior work, we explored the problem of estimating aggregate query results over dirty data [10,16].”
• SampleClean: Fast and Reliable Analytics on Dirty Data (2015, background): “View Cleaning [30]: View Cleaning generalizes the notion of data cleaning to include expensive incremental maintenance of out-of-date views.”
• PrivateClean: Data Cleaning and Differential Privacy (2016, background): “This is a tradeoff that has been highlighted before in the SampleClean project [23,44].”
• ActiveClean: Interactive Data Cleaning While Learning Convex Loss Models (2016, background): “Krishnan et al. explored how samples of materialized views can be maintained similar to how models are updated with a sample of clean data in this work [24].”
• Data Cleaning: Overview and Emerging Challenges (2016, background): “This problem was further extended to study aggregate queries on materialized views [50].”
• Towards reliable interactive data cleaning: a user survey and recommendations (2016, background): “…of aspects of this problem, including reducing the latency of human operations [20], understanding quality in macrotasks [18], reducing latency with sampling [29], mitigating cognitive biases [27], addressing privacy concerns [30], and defining statistical correctness for data cleaning [28] [31].”