Exploratory Data Mining and Data Cleaning

By Tamraparni Dasu, Theodore Johnson. 2003

612 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/f147db1429c934b3578d929c5e3f3d2ae97d1382

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

Citations in the corpus

Trends in Cleaning Relational Data: Consistency and Deduplication (2015, background): “Bertossi [8] provides complexity results for repairing inconsistent data, and performing consistent query answering on inconsistent data; Fan and Geerts [44] discuss the use of data quality rules in data consistency, data currency, and data completeness, how different aspects of data quality issues ”
A sample-and-clean framework for fast and accurate query processing on dirty data (2014, background): “Data Cleaning: There have been many studies on various data-cleaning techniques, such as rule-based approaches [17, 23], outlier detection [18,31], filling missing values [48,52], and duplicate detection [9,12].”
Hyperspectral measurements of yellow rust and fusarium head blight in cereal crops: Part 2: On-line field measurement (2018, methodology): “Although pre-processing of spectral data includes techniques such as 176 smoothing, if the process of cleaning the data is intensive due to noisy spectra it can lead to the loss of 177 important spectral features, and thus impact on the success of analysis (Dasu & Johnson, 2003).”
Wrangler: interactive visual specification of data transformation scripts (2011, neutral): “data warehousing projects [4].”
The Perils of Ignoring Data Suitability - The Suitability of Data used to Train Neural Networks Deserves More Attention (2011, background): “is good and (Dasu and Johnson, 2003) has some useful content but such books are rare compared to the number of data mining and neural network algorithm books on the market.”
SampleClean: Fast and Reliable Analytics on Dirty Data (2015, background): “Data Cleaning: There have been many studies on various data-cleaning techniques, such as rule-based approaches [21, 18], outlier detection [26, 19], ﬁlling missing values, and duplicate detection [13, 9, 44].”
Quantitative Data Cleaning for Large Databases (2008, methodology): “Exploratory Data Analysis [Tukey, 1977] (sometimes called Exploratory Data Mining in more recent literature [Dasu and Johnson, 2003]) typically involves a human in the process of understanding properties of a dataset, including the identification and possible rectification of errors.”
ActiveClean: Interactive Data Cleaning For Statistical Modeling (2016, background): “In the data cleaning literature, error detection and error repair are treated as two distinct problems [9,13,30].”
Methodologies for data quality assessment and improvement (2009, positive): “Techniques for error localization and correction have been proposed for inconsistencies, incomplete data, and outliers [Dasu and Johnson 2003]; [Batini and Scannapieco 2006].”
oddball: Spotting Anomalies in Weighted Graphs (2010, background): “In addition to revealing suspicious, illegal and/or dangerous behavior, anomaly detection is useful for spotting rare events, as well as for the thankless, but absolutely vital task of data cleansing [12].”
Query-Oriented Data Cleaning with Oracles (2015, background): “As previously mentioned, extensive research has also been devoted to develop algorithms to ensure the quality of answers, both for individual answers (e.g., outlier detection [42, 59, 16]) and aggregated answers (e.g., using error probability, or an average weighted by trust [48, 47, 54]).”
A literature review on the application of evolutionary computing to credit scoring (2013, background): “The primary aim is to derive a subset of data, which is expected to be a representative sample of the problem to be modelled (Pyle, 1999; Dasu and Johnson, 2003).”
Profiler: integrated statistical analysis and visualization for data quality assessment (2012, background): “The database and statistics literature includes many taxonomies of anomalous data [5, 7, 10, 19, 24, 33].”
Towards reliable interactive data cleaning: a user survey and recommendations (2016, background): “Over the last two decades, data cleaning has been a key area of database research (see surveys [14], [34], and tutorial [11]).”
Research directions in data wrangling: Visualizations and transformations for usable and credible data (2011, neutral): “Others estimate that data cleaning accounts for up to 80% of the development time and cost in data warehousing projects.(1) Often this process requires writing idiosyncratic scripts in programming languages such as Python, Perl, and R, or engaging in tedious manual editing using tools such as Micros”