Detecting Data Errors: Where are we and what needs to be done?

By Ziawasch Abedjan, Xu Chu, Dong Deng, Raul Castro Fernandez, Ihab F. Ilyas, Mourad Ouzzani, Paolo Papotti, M. Stonebraker, Nan Tang. 2016

300 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

Citations in the corpus

BoostClean: Automated Error Detection and Repair for Machine Learning (2017, methodology): “[5], we found that a purely quantitative approach does not perform well in comparison”
HoloClean: Holistic Data Repairs with Probabilistic Inference (2017, background): “datasets [4], this is not the case with automatic data repairing [30].”
Towards Automated Data Cleaning Workflows (2019, methodology): “As metadata provides signals on individual values and inter-column relationships, they support the four major types of error detection strategies: pattern violation detection , rule violation detection , outlier detection , and duplicate-based error detection [1].”
Challenges in Deploying Machine Learning: A Survey of Case Studies (2020, background): “As mentioned in Section 3, data integrity plays a big part in quality control, and is an active branch of research [18].”
A Data Quality Metric (DQM): How to Estimate the Number of Undetected Errors in Data Sets (2016, methodology): “Most practical data cleaning tools rely on humans to verify the errors or repairs [2], and Crowd-based cleaning techniques use humans to identify errors in conjunction with al-”
The Data Civilizer System (2017, background): “We have recently studied several representative data cleaning systems on a collection of real world data sets “from the wild” [4] and we found out that there was no cleaning Esperanto.”
Data Integration: The Current Status and the Way Forward (2018, unknown): “However, it is not clear how to integrate these tools in a scalable way in workﬂows like the ones provisioned by Tamr , and in which order these cleaning tools should be used when interleaved with the main integration components [1].”