SLiMFast: Guaranteed Results for Data Fusion and Source Reliability

By Theodoros Rekatsinas, Manas R. Joglekar, Hector Garcia-Molina, Aditya G. Parameswaran, Christopher R'e. 2015

63 citations recorded by the source corpus.

We focus on data fusion, i.e., the problem of unifying conflicting data from data sources into a single representation by estimating the source accuracies. We propose SLiMFast, a framework that expresses data fusion as a statistical learning problem over discriminative probabilistic models, which in many cases correspond to logistic regression. In contrast to previous approaches that use complex generative models, discriminative models make fewer distributional assumptions over data sources and allow us to obtain rigorous theoretical guarantees. Furthermore, we show how SLiMFast enables incorporating domain knowledge into data fusion, yielding accuracy improvements of up to 50% over state-of-the-art baselines. Building upon our theoretical results, we design an optimizer that obviates the need for users to manually select an algorithm for learning SLiMFast's parameters. We validate our optimizer on multiple real-world datasets and show that it can accurately predict the learning algorithm that yields the best data fusion results.

Subfields: Artificial Intelligence, Information Retrieval & Search, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

In Signal Processing

Citations in the corpus

Snorkel: Rapid Training Data Creation with Weak Supervision (2017, background): “Related Data Management Problems: Researchers have considered related problems in data management, such as data fusion [15,40] and truth discovery [28].”
HoloClean: Holistic Data Repairs with Probabilistic Inference (2017, background): “This allows Holo-Clean to reason about the trustworthiness of different sources [35] to obtain more accurate repairs.”
Snuba: Automating Weak Supervision to Label Training Data (2018, unknown): “Areas like data fusion [13, 41, 38] and truth discovery [30] also look at the problem of estimating how reliable different data sources are while utilizing probabilistic graphical models like Snuba.”
Ten Years of WebTables (2018, background): “Even so, there is still opportunity to develop better tools for structured extraction [42, 11], data cleaning [28], and objectivity-testing [38, 29].”
Cleaning Crowdsourced Labels Using Oracles For Statistical Classification (2018, background): “There are three research topics in crowdsourcing related to our work: task assignment [4, 6, 13, 17, 26, 31, 46], truth inference [12,16,19,33,45], and active learning (from crowds) [18, 24, 28, 43].”