Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation

By Qi Li, Yaliang Li, Jing Gao, Bo Zhao, Wei Fan, Jiawei Han. 2014

507 citations recorded by the source corpus.

In many applications, one can obtain descriptions about the same objects or events from a variety of sources. As a result, this will inevitably lead to data or information conflicts. One important problem is to identify the true information (i.e., the truths) among conflicting sources of data. It is intuitive to trust reliable sources more when deriving the truths, but it is usually unknown which one is more reliable a priori. Moreover, each source possesses a variety of properties with different data types. An accurate estimation of source reliability has to be made by modeling multiple properties in a unified model. Existing conflict resolution work either does not conduct source reliability estimation, or models multiple properties separately. In this paper, we propose to resolve conflicts among multiple sources of heterogeneous data types. We model the problem using an optimization framework where truths and source reliability are defined as two sets of unknown variables. The objective is to minimize the overall weighted deviation between the truths and the multi-source observations where each source is weighted by its reliability. Different loss functions can be incorporated into this framework to recognize the characteristics of various data types, and efficient computation approaches are developed. Experiments on real-world weather, stock and flight data as well as simulated multi-source data demonstrate the necessity of jointly modeling different data types in the proposed framework.

Subfields: Global and Planetary Change, Management Science and Operations Research, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Global and Planetary Change

Jiawei Han

In Management Science and Operations Research

In Signal Processing

Citations in the corpus

Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer (2019, methodology): “There exist a long chain of recent poisoning attacks and defenses for federated learning [37], [55], [57], [13], [53], [34], [40], [12], [10].”
SLiMFast: Guaranteed Results for Data Fusion and Source Reliability (2015, background): “…collaborators from a medical school—who are currently engaged in extracting information from scientific articles to populate a structured data repository—we find that using data fusion can be challenging for users despite the numerous approaches proposed in the literature [9, 14, 21, 22, 28, 38].”
Toward Annotator Group Bias in Crowdsourcing (2021, unknown): “…et al. (2012); Aydin et al. (2014) use worker probability to model the ability of an annotator to correctly answer a task, and some other works (Whitehill et al., 2009; Li et al., 2014b) introduce a similar concept, worker quality, by changing the value range from [0 , 1] to ( −∞ , + ∞ ) .”
ExFaKT: A Framework for Explaining Facts over Knowledge Graphs and Text (2019, background): “[11, 17, 21, 24, 25, 31]) perform joint estimation of the fact’s truth value based on the credibility of the fact candidates syntactically spotted in text and the trustworthiness of the underlying sources.”
A Survey on Truth Discovery (2015, background): “Some truth discovery methods [20, 26, 32, 53, 56] have the source independence assumption, which can be interpreted as the fact that sources make their observations independently instead of copying from each other.”
A Confidence-Aware Approach for Truth Discovery on Long-Tail Data (2014, methodology): “The baseline methods include some state-of-the-art truth discovery methods: GTM [36], TruthFinder [34], AccuSim [8], Investment [23], 3-Estimates [16], and CRH [18].”
An Overview of Federated Deep Learning Privacy Attacks and Defensive Strategies (2020, methodology): “[74] proposed a weighting strategy by reliability for the use of Heterogeneous Data.”