Improving Data Quality in Data Warehousing Applications

By Lin Li, Taoxin Peng, J. Kennedy. 2010

3 citations recorded by the source corpus.

There is a growing awareness that high quality of data is a key to today’s business success and dirty data that exits within data sources is one of the reasons that cause poor data quality. To ensure high quality, enterprises need to have a process, methodologies and resources to monitor and analyze the quality of data, methodologies for preventing and/or detecting and repairing dirty data. However in practice, detecting and cleaning all the dirty data that exists in all data sources is quite expensive and unrealistic. The cost of cleaning dirty data needs to be considered for most of enterprises. Therefore conflicts may arise if an organization intends to clean their data warehouses in that how do they select the most important data to clean based on their business requirements. In this paper, business rules are used to classify dirty data types based on data quality dimensions. The proposed method will be able to help to solve this problem by allowing users to select the appropriate group of dirty data types based on the priority of their business requirements. It also provides guidelines for measuring the data quality with respect to different data quality dimensions and also will be helpful for the development of data cleaning tools.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Sanjay Krishnan
2. M. Franklin
3. Eugene Wu
4. Ken Goldberg
5. Jiannan Wang
6. Ihab F. Ilyas
7. Yinjun Wu
8. Xu Chu

Citations in the corpus

• BoostClean: Automated Error Detection and Repair for Machine Learning (2017, background): “Numerous data quality surveys across the database, statistics, and scientific literature highlight the prevalence, variety, and importance of this class of errors, which include missing data, incorrect data, or inconsistent representations of the same logical data value [25,29,33,39].”
• Research directions in data wrangling: Visualizations and transformations for usable and credible data (2011, unknown): “Li et al.(11) outline 41 different types of dirty data, and examine the costs of fixing these errors within different contexts.”