Fusing data with correlations

By R. Pochampally, Anish Das Sarma, Xin Luna Dong, Alexandra Meliou, Divesh Srivastava. 2014

135 citations recorded by the source corpus.

Many applications rely on Web data and extraction systems to accomplish knowledge-driven tasks. Web information is not curated, so many sources provide inaccurate, or conflicting information. Moreover, extraction systems introduce additional noise to the data. We wish to automatically distinguish correct data and erroneous data for creating a cleaner set of integrated data. Previous work has shown that a naive voting strategy that trusts data provided by the majority or at least a certain number of sources may not work well in the presence of copying between the sources. However, correlation between sources can be much broader than copying: sources may provide data from complementary domains (negative correlation), extractors may focus on different types of information (negative correlation), and extractors may apply common rules in extraction (positive correlation, without copying). In this paper we present novel techniques modeling correlations between sources and applying it in truth finding. We provide a comprehensive evaluation of our approach on three real-world datasets with different characteristics, as well as on synthetic data, showing that our algorithms outperform the existing state-of-the-art techniques.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research, Statistics and Probability, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

In Statistics and Probability

In Information Extraction

Christopher R'e

Citations in the corpus

Snorkel: Rapid Training Data Creation with Weak Supervision (2017, background): “Work on data fusion has also explored how to model user-specified correlations among data sources [36].”
A Survey on Truth Discovery (2015, background): “Similarly, in [35], the authors also calculate both the source precision and recall to discover multiple truths.”
Snorkel: rapid training data creation with weak supervision (2019, background): “Work on data fusion has also explored how to model user-speciﬁed correlations among data sources [36].”
Snuba: Automating Weak Supervision to Label Training Data (2018, unknown): “Areas like data fusion [13, 41, 38] and truth discovery [30] also look at the problem of estimating how reliable different data sources are while utilizing probabilistic graphical models like Snuba.”