Data Cleansing: Beyond Integrity Analysis

By Jonathan I. Maletic, Andrian Marcus. 2000

234 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

Citations in the corpus

Learning kernel logistic regression in the presence of class label noise (2014, background): “The seemingly straightforward approach is by means of data preprocessing where any suspect samples are removed or relabelled [7,1,29,37,31,18].”
Toward Scalable Systems for Big Data Analytics: A Technology Tutorial (2014, background): “Moreover, format checks, completeness checks, reasonableness checks, and limit checks [105] are normally considered in the cleansing process.”
Class Noise vs. Attribute Noise: A Quantitative Study (2003, background): “However, errors in a large dataset are common and severe, and unless an organization takes extreme measures in an effort to avoid data errors, the field error rates are typically around 5% or more (Wu 1995; Orr 1998; Maletic and Marcus 2000).”
Big Data: A Survey (2014, methodology): “procedures [68]: defining and determining error types, searching and identifying errors, correcting errors, doc-”
Classification in the Presence of Label Noise: A Survey (2014, background): “Real-world databases are estimated to contain around ﬁve percents of encoding errors, all ﬁelds taken together, when no speciﬁc measures are taken [59]–[61].”
A Survey of Indexing Techniques for Scalable Record Linkage and Deduplication (2012, background): “While statisticians and health researchers commonly name the task of matching records as data or record linkage [11] (the term used in this paper), the computer science and database communities refer to the same process as data or field matching [12], data integration [13], data scrubbing or cleanin”
Multi-class classification in the presence of labelling errors (2011, background): “Most often label noise is dealt with in an ad-hoc heuristic manner, and it involves some preprocessing of the training set for data removal or relabelling [6].”
Dealing with noise in defect prediction (2011, background): “Some studies show that errors in a large dataset are common and field error rates are typically around 5% or more [29, 12].”
Global-Local GCN: Large-Scale Label Noise Cleansing for Face Recognition (2020, background): “Label noise cleansing is a long-term issue in building up a dataset [9, 10, 24, 25, 30, 32].”
Conditional functional dependencies for capturing data inconsistencies (2008, unknown): “Statistical and mining methods have also been used for detecting inconsistencies, e.g., outlier detection (see [ Maletic and Marcus 2000 ] for a survey).”