ActiveClean: Interactive Data Cleaning While Learning Convex Loss Models

By Sanjay Krishnan, Jiannan Wang, Eugene Wu, M. Franklin, Ken Goldberg. 2016

29 citations recorded by the source corpus.

Data cleaning is often an important step to ensure that predictive models, such as regression and classification, are not affected by systematic errors such as inconsistent, out-of-date, or outlier data. Identifying dirty data is often a manual and iterative process, and can be challenging on large datasets. However, many data cleaning workflows can introduce subtle biases into the training processes due to violation of independence assumptions. We propose ActiveClean, a progressive cleaning approach where the model is updated incrementally instead of re-training and can guarantee accuracy on partially cleaned data. ActiveClean supports a popular class of models called convex loss models (e.g., linear regression and SVMs). ActiveClean also leverages the structure of a user's model to prioritize cleaning those records likely to affect the results. We evaluate ActiveClean on five real-world datasets UCI Adult, UCI EEG, MNIST, Dollars For Docs, and WorldBank with both real and synthetic errors. Our results suggest that our proposed optimizations can improve model accuracy by up-to 2.5x for the same amount of data cleaned. Furthermore for a fixed cleaning budget and on all real dirty datasets, ActiveClean returns more accurate models than uniform sampling and Active Learning.

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sanjay Krishnan
2. Eugene Wu
3. Ihab F. Ilyas
4. M. Franklin
5. Jiannan Wang
6. Ken Goldberg
7. Xu Chu
8. Daniel Haas

In Management Science and Operations Research

1. Sanjay Krishnan
2. Eugene Wu
3. M. Franklin
4. Jiannan Wang
5. Ihab F. Ilyas
6. Ken Goldberg
7. Xu Chu
8. Daniel Haas

Citations in the corpus

• ActiveClean: An Interactive Data Cleaning Framework For Modern Machine Learning (2016, methodology): “The analyst initializes an ActiveClean with an ML model, a featurization function, and the base data, and the ActiveClean initially returns the model trained on the dataset.”
• Towards reliable interactive data cleaning: a user survey and recommendations (2016, background): “…of aspects of this problem, including reducing the latency of human operations [20], understanding quality in macrotasks [18], reducing latency with sampling [29], mitigating cognitive biases [27], addressing privacy concerns [30], and defining statistical correctness for data cleaning [28] [31].”
• Data Cleaning: Overview and Emerging Challenges (2016, methodology): “For example, in one fraud prediction example, simply merging inconsistent attributes before SVM model training improved true positive detection probabilities from 62% to 91% [52].”
• ActiveClean: Interactive Data Cleaning For Statistical Modeling (2016, methodology): “In this generalization, the C(·) function is composed of schema preserving map and filter operations applied to the entire dataset.”