HoloClean: Holistic Data Repairs with Probabilistic Inference

By Theodoros Rekatsinas, Xu Chu, Ihab F. Ilyas, Christopher R'e. 2017

582 citations recorded by the source corpus.

We introduce HoloClean, a framework for holistic data repairing driven by probabilistic inference. HoloClean unifies qualitative data repairing, which relies on integrity constraints or external data sources, with quantitative data repairing methods, which leverage statistical properties of the input data. Given an inconsistent dataset as input, HoloClean automatically generates a probabilistic program that performs data repairing. Inspired by recent theoretical advances in probabilistic inference, we introduce a series of optimizations which ensure that inference over HoloClean's probabilistic model scales to instances with millions of tuples. We show that HoloClean finds data repairs with an average precision of ∼ 90% and an average recall of above ∼ 76% across a diverse array of datasets exhibiting different types of errors. This yields an average F1 improvement of more than 2× against state-of-the-art methods.

Subfields: Artificial Intelligence, Management Science and Operations Research, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In ML Systems & Hardware

Citations in the corpus

Snorkel: Rapid Training Data Creation with Weak Supervision (2017, background): “Other forms include crowdsourced labels [37, 50], rules and heuristics for labeling data [39, 52], and others [29, 30, 30, 46, 51].”
Benchmark and Survey of Automated Machine Learning Frameworks (2019, background): “Yet, most current approaches still aim to assist a human data scientist instead of fully automated data cleaning, (e.g., Krishnan et al., 2015; Khayyaty et al., 2015; Krishnan et al., 2016; Eduardo & Sutton, 2016; Rekatsinas et al., 2017).”
AlphaClean: Automatic Generation of Data Cleaning Pipelines (2019, background): “cleaning [4, 23, 28, 30, 41, 45] has revealed human-time — Machine Learning in Data Cleaning: Machine learning Application-Aware Cleaning: Semantics about the down-stream Generating Cleaning Programs: A composable data”
A Survey on Data Collection for Machine Learning: A Big Data - AI Integration Perspective (2018, background): “HoloClean [158] is a state-of-art data cleaning system that uses quality rules, value correlations, and reference”
Towards Automated Data Cleaning Workflows (2019, background): “This classiﬁcation model provides the user with those features that correlate with the corresponding error and therefore gives the user an idea of the context that this error occurs in. Figure 3 shows an example of this approach on Flights data.”
Data Integration: The Current Status and the Way Forward (2018, background): “There is promising work in this area, such as the HoloClean project [12], which uses statistical inference to impute missing values and to suggest repairs for erroneous cells.”
An Empirical Analysis of Backward Compatibility in Machine Learning Systems (2020, background): “Data cleaning is a topic of long-term interest in the data management and mining community [8, 33, 35].”
The Role of Massively Multi-Task and Weak Supervision in Software 2.0 (2019, background): “…and Empirical Performance: Modern machine learning models have demonstrated state-of-the-art accuracies on a broad range of tasks, including both traditionally machine learning-driven ones like search and translation, and traditionally heuristic ones like data cleaning and integration [35].”
Data Cleaning for Accurate, Fair, and Robust Models: A Big Data - AI Integration Approach (2019, background): “mework can be extended to more general combinations of data sanitization, data cleaning, and unfairness mitigation. Beyond removing duplicates, data cleaning can be any general process like HoloClean [11]. Data sanitization can also employ more sophisticated defenses [9]. Of course, one should caref”
Ten Years of WebTables (2018, background): “Even so, there is still opportunity to develop better tools for structured extraction [42, 11], data cleaning [28], and objectivity-testing [38, 29].”
Machine Learning for Synthetic Data Generation: a Review (2023, methodology): “[177] applies data cleaning method [186] to fix the violations on the structure of the data in the synthetic data.”
Automating Large-Scale Data Quality Verification (2018, background): “HoloClean generates a probabilistic model over a dataset that combines integrity constraints and external data sources to generate data repair suggestions [35].”
Data collection and quality challenges in deep learning: a data-centric AI perspective (2021, methodology): “2 Tabular & Image Data-only cleaning [81,130]”
Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale (2018, background): “[24] proposed HoloClean, which uses weakly supervised machine learning to learn to correct errors.”
Can Foundation Models Wrangle Your Data? (2022, methodology): “Finally, for error detection, we evaluate on the benchmark Hospital dataset that is used across several data cleaning papers [21, 34, 71].”
Fidelity-Weighted Learning (2017, methodology): “For instance, methods that use a generative model to correct weak labels such that a discriminative model can be trained more effectively (Ratner et al., 2016; Rekatsinas et al., 2017; Varma et al., 2017).”