Scorpion: Explaining Away Outliers in Aggregate Queries

By Eugene Wu, Samuel Madden. 2013

262 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

AlphaClean: Automatic Generation of Data Cleaning Pipelines (2019, methodology): “Recent work such as ActiveClean [31] extend this work to downstream machine learning applications, while Scorpion [46] uses the visualization-specified errors to search for approximate deletion transformations.”
BoostClean: Automated Error Detection and Repair for Machine Learning (2017, methodology): “Sensor: The Intel sensor dataset [4,49,50] contains 928,991 temperature, humidity, and light sensor readings a sensor deployment.”
Data X-Ray: A Diagnostic Tool for Data Errors (2015, background): “The Scorpion system [56] ﬁnds predicates on the input data as explanations for a labeled set of outlier points in an aggregate query over a single relation .”
SampleClean: Fast and Reliable Analytics on Dirty Data (2015, background): “While complementary, one major difference between the three works [33, 46, 11] and View Cleaning is that they require an explicit speciﬁcation of erroneous rows in a materialized view.”
Effective Data Cleaning with Continuous Evaluation (2016, methodology): “The work in [18, 4] provides mechanisms to compute the likelihood of errors in the source data, given the lineage and the type of errors detected at the reports level.”
Combining Design and Performance in a Data Visualization Management System (2017, background): “Visualization Explanation: outlier explanation techniques [44, 59] rely on the ability to trace outliers in the visualization back to their inputs in order to identify the inputs to the explanation generation algorithms.”
Data Cleaning: Overview and Emerging Challenges (2016, background): “FDsvaluemodiﬁcation[13] Holisticdatacleaning[22] CrowdER[74] Corleone[35] CausalityAnalysis[58] Scorpion[79] DBRx[16] Table 1: A sample of qualitative error detection techniques.”
Trends in Cleaning Relational Data: Consistency and Deduplication (2015, background): “Techniques for error propagation vary according to the type of data transformations assumed, such as Boolean expressions [87], aggregation on numerical attributes [124, 123], and more general SPJA queries [19].”
PALM: Machine Learning Explanations For Iterative Debugging (2017, result): “This notion is similar to responsibility in the data lineage and explanation literature [4, 17], in the context of black-box machine learning models as opposed to database queries with well deﬁned semantics.”
Detecting Data Errors: Where are we and what needs to be done? (2016, background): “• Quantitative error detection algorithms that expose outliers, and glitches in the data [2, 9, 28,32,34].”
MacroBase: Analytic Monitoring for the Internet of Things (2016, methodology): “In contrast, training a decision tree based on inliers and outliers (maximum depth: 10) as recently suggested by Scorpion [49] yields an average summarization time approximately 11.5x slower than MacroBase.”
MacroBase: Prioritizing Attention in Fast Data (2016, methodology): “We ran MDP over a synthetic dataset generated in line with those used to evaluate recent anomaly detection systems [68,80].”
Outlier Detection in Heterogeneous Datasets using Automatic Tuple Expansion (2016, background): “Research has been done to attempt to explain why outliers exist given properties of the original data [22].”
Stale View Cleaning: Getting Fresh Answers from Stale Materialized Views (2015, methodology): “The sampling technique proposed in Section 4 of our paper could be used to approximate the data cleaning techniques in [6,36,50] and this is an exciting avenue of future work.”