Holistic data cleaning: Putting violations into context

By Xu Chu, Ihab F. Ilyas, Paolo Papotti. IEEE International Conference on Data Engineering, 2013

375 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/c4221a899528798105ca94e509027e7210a87d6b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

Citations in the corpus

“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI (2021, neutral): “Data cleaning and wrangling systems address data quality issues by using integrity constraints [27], type inference [36], schema matching [43], outlier detection [51] and more.”
Can Foundation Models Wrangle Your Data? (2022, methodology): “Traditionally, ED systems have been heavily reliant on rule-based algorithms which enforce data constraints through functional dependencies or knowledge bases [23, 24, 26].”
Data Lifecycle Challenges in Production Machine Learning (2018, background): “In the literature, cost-based models [16, 42] and conﬂict hypergraphs [23] are proposed.”
Trends in Cleaning Relational Data: Consistency and Deduplication (2015, methodology): “We discuss three examples techniques: holistic data cleaning [26] is an example technique for automatically detecting violations of ICs; CrowdER [114] uses a hybrid machine-human approach to improve data deduplication accuracy; and Corleone [59] is a data deduplication”
Data Cleaning: Overview and Emerging Challenges (2016, methodology): “…there has been a surge of interest from both industry and academia on different aspects of data cleaning including new abstractions [10, 30, 77, 22, 33, 14, 73], interfaces [1, 26], approaches for scalability [75, 45, 3, 53, 67], and crowdsourcing techniques [35, 69, 62, 18, 82, 23, 37, 60,…”
Detecting Data Errors: Where are we and what needs to be done? (2016, methodology): “We use the tool DC-Clean that focuses on denial constraints (DCs) [6] and DCs subsume the majority of the commonly used constraint languages.”
KATARA: A Data Cleaning System Powered by Knowledge Bases and Crowdsourcing (2015, background): “A plethora of data cleaning approaches that are based on integrity constraints [2,7,9,20,36], statistics [30], or machine learning [43], have been proposed in the past.”
Towards Automated Data Cleaning Workflows (2019, methodology): “There has already been a large body of work on data cleaning [4, 6, 23, 5].”
Data X-Ray: A Diagnostic Tool for Data Errors (2015, background): “Again, there is a large arsenal of tools in this category that use rules [7, 14] and functional dependencies [12, 26], while there is also work that focuses on generating repairs in an interactive fashion, using user”
Effective Data Cleaning with Continuous Evaluation (2016, background): “Not surprisingly, developing data quality solutions is a challenging topic and is rich with deep theoretical and engineering problems.”
HoloClean: Holistic Data Repairs with Probabilistic Inference (2017, methodology): “To evaluate data repairing on these datasets, we manually labeled 2,000 and 2,500 cells, respectively: We focused on tuples identiﬁed as erroneous by the error detection mechanisms of Holistic, KATARA, and SCARE.”
Discovering Denial Constraints (2013, methodology): “Specifically, Σg for Tax dataset has 8 DCs; Σg for Hospital is retrieved from [10] and has 7 DCs; and Σg for SP Stock has 6 DCs.”