Provenance management in curated databases

By Peter Buneman, Adriane P. Chapman, J. Cheney. SIGMOD Conference, 2006

354 citations recorded by the source corpus.

Curated databases in bioinformatics and other disciplines are the result of a great deal of manual annotation, correction and transfer of data from other sources. Provenance information concerning the creation, attribution, or version history of such data is crucial for assessing its integrity and scientific value. General purpose database systems provide little support for tracking provenance, especially when data moves among databases. This paper investigates general-purpose techniques for recording provenance for data that is copied among databases. We describe an approach in which we track the user's actions while browsing source databases and copying data into a curated database, in order to record the user's actions in a convenient, queryable form. We present an implementation of this technique and use it to evaluate the feasibility of database support for provenance management. Our experiments show that although the overhead of a naive approach is fairly high, it can be decreased to an acceptable level using simple optimizations.

Subfields: Computer Networks and Communications, Information Systems, Information Systems and Management

View Paper: https://www.semanticscholar.org/paper/0fd7afe76412f1101b1f8eec38c3969e09a98a79

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Amol Deshpande
2. Hui Miao
3. Amit Chavan
4. J. Cheney
5. Wang Chiew Tan
6. Laura Chiticariu
7. Aad P. A. van Moorsel
8. Jennifer Widom

In Information Systems

1. Amol Deshpande
2. Hui Miao
3. Amit Chavan
4. Wang Chiew Tan
5. Aad P. A. van Moorsel

In Information Systems and Management

1. Amol Deshpande
2. Hui Miao
3. Amit Chavan
4. Wang Chiew Tan
5. J. Cheney
6. Laura Chiticariu
7. Jennifer Widom

Citations in the corpus

• The relationship between trust in AI and trustworthy machine learning technologies (2019, background): “It involves proposals on how to store provenance data [18, 20, 21, 74, 94], summarisation of provenance data [2, 28, 66], specific query language for the provenance data [28], query explainability [8, 10], Natural language processing for prove-”
• ProvDB: A System for Lifecycle Management of Collaborative Analysis Workflows (2016, background): “On the other hand, in dataflow systems where the operators are written in a declarative language (e.g., SQL, Pig Latin, Spark), data provenance at record level can be captured if needed [5,6,8,10,17,22,31,53].”
• Provenance in Databases: Why, How, and Where (2009, background): “Recently, various schemes aimed at reducing the amount of extra information stored have been investigated in [31] [38] [54], and the problem of compressing or approximating provenance has been explored in [9] [16] [50].”