Profiling relational data: a survey
By Ziawasch Abedjan, Lukasz Golab, Felix Naumann. The VLDB journal, 2015
348 citations recorded by the source corpus.
Profiling data to determine metadata about a given dataset is an important and frequent activity of any IT professional and researcher and is necessary for various use-cases. It encompasses a vast array of methods to examine datasets and produce metadata. Among the simpler results are statistics, such as the number of null values and distinct values in a column, its data type, or the most frequent patterns of its data values. Metadata that are more difficult to compute involve multiple columns, namely correlations, unique column combinations, functional dependencies, and inclusion dependencies. Further techniques detect conditional properties of the dataset at hand. This survey provides a classification of data profiling tasks and comprehensively reviews the state of the art for each class. In addition, we review data profiling tools and systems from research and industry. We conclude with an outlook on the future of data profiling beyond traditional profiling tasks and beyond relational databases.
Subfields: Computer Networks and Communications, Information Systems, Management Science and Operations Research
View Paper: https://www.semanticscholar.org/paper/326d79ced627591a1710296d7868b832a2e57367
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Networks and Communications
- Ihab F. Ilyas
- Xu Chu
- Ziawasch Abedjan
- Paolo Papotti
- M. Stonebraker
- Samuel Madden
- Nan Tang
- Philipp Schmidt
In Information Systems
- Ihab F. Ilyas
- Xu Chu
- Ziawasch Abedjan
- Paolo Papotti
- Mohammad Mahdavi
- Larysa Visengeriyeva
- Felix Neutatz
- Nan Tang
In Management Science and Operations Research
- Ihab F. Ilyas
- Xu Chu
- Ziawasch Abedjan
- Paolo Papotti
- Nan Tang
- Mohammad Mahdavi
- Larysa Visengeriyeva
- Felix Neutatz
Citations in the corpus
- Towards Automated Data Cleaning Workflows (2019, methodology): “We have created a dirtiness profile based on data profiling features [2].”
- The Data Civilizer System (2017, background): “Our profiles also contain information about data cardinality, data type, and numerical ranges if applicable [5].”
- Detecting Data Errors: Where are we and what needs to be done? (2016, background): “• Quantitative error detection algorithms that expose outliers, and glitches in the data [2, 9, 28,32,34].”
- Automating Large-Scale Data Quality Verification (2018, methodology): “Our constraint suggestion functionality is built on a heuristics-based approach employing single-column profiling [1].”