Incremental Knowledge Base Construction Using DeepDive

By Christopher De Sa, Alexander J. Ratner, Christopher R'e, Jaeho Shin, Feiran Wang, Sen Wu, Ce Zhang. 2015

295 citations recorded by the source corpus.

Populating a database with information from unstructured sources—also known as knowledge base construction (KBC)—is a long-standing problem in industry and research that encompasses problems of extraction, cleaning, and integration. In this work, we describe DeepDive, a system that combines database and machine learning ideas to help develop KBC systems, and we present techniques to make the KBC process more efficient. We observe that the KBC process is iterative, and we develop techniques to incrementally produce inference results for KBC systems. We propose two methods for incremental inference, based, respectively, on sampling and variational techniques. We also study the trade-off space of these methods and develop a simple rule-based optimizer. DeepDive includes all of these contributions, and we evaluate DeepDive on five KBC systems, showing that it can speed up KBC inference tasks by up to two orders of magnitude with negligible impact on quality.

Subfields: Artificial Intelligence, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Concrete Problems in AI Safety (2016, background): “Better yet, if we have a model, we can extrapolate forward and ask whether an action will take us outside the safe state space.”
Data Programming: Creating Large Training Sets, Quickly (2016, background): “There is also a wealth of examples where additional heuristic patterns used to label training data are collected from unlabeled data [6] or directly from users [19,27], in a similar manner to our approach, but without a framework to deal with the fact that said labels are explicitly noisy.”
HoloClean: Holistic Data Repairs with Probabilistic Inference (2017, methodology): “A probabilistic model in DeepDive corresponds to a collection of rules in DDlog.”
The Data Civilizer System (2017, background): “…[21] mainly performs schema mappings and record linkage, NADEEF [10] and BigDansing [17] provide generic programming interface for users to deﬁne their data quality rules, and DeepDive [20] extracts facts and structured information from large corpora of text, images and other unstructured sources.”
Inferring Generative Model Structure with Static Analysis (2017, background): “As the need for labeled training data grows, a common alternative is to utilize weak supervision sources such as distant supervision [10, 24], multi-instance learning [16, 30], and heuristics [8, 35].”
Fonduer: Knowledge Base Construction from Richly Formatted Data (2017, unknown): “Multimodality Classical KBC systems model input data as unstructured text [23, 26, 36].”
MacroBase: Prioritizing Attention in Fast Data (2016, methodology): “ms or “interesting”-ness metrics used in computation. Instead, users simply describe their data sources— speciﬁcally, the set of attributes and metrics that are of interest. As in the DeepDive system [43], this allows users to think about their data, not about the algorithms behind MacroBase. When s”
Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis (2017, methodology): “Fries also utilizes Stanford’s DeepDive application [55] for structured relationships and predictions.”
Robust Discovery of Positive and Negative Rules in Knowledge Bases (2018, methodology): “We compare the output of DeepDive upon its spouse example trained with different sets of negative examples over two datasets.”
SLiMFast: Guaranteed Results for Data Fusion and Source Reliability (2015, methodology): “SLiMFast, Sources-ERM and SourcesEM, are implemented over DeepDive v0.”
MacroBase: Analytic Monitoring for the Internet of Things (2016, methodology): “As in the DeepDive system [43], this allows users to think about their data, not about the algorithms behind MacroBase.”
Socratic Learning: Augmenting Generative Models to Incorporate Latent Subsets in Training Data (2016, background): “Using weak supervision to eﬃciently label training data is a popular approach, relying on methods like distant supervision [5, 18], multi-instance learning [22, 13], heuristic patterns [11, 4], user input [25, 26, 19], and crowdsourcing [9].”
Learning the Structure of Generative Models without Labeled Data (2017, methodology): “Previous methods that support heuristics for weak supervision (e.g., Bunescu & Mooney, 2007; Shin et al., 2015) do not model the noise inherent in these sources.”
DeepDive: Declarative Knowledge Base Construction (2016, methodology): “To automatically choose the materialization strategy, we developed a simple rule-based optimizer [39].”
Data programming with DDLite: putting humans in a different part of the loop (2016, methodology): “After implementing these improvements, we found that constructing large, labeled training data sets had become the primary development bottleneck.”
SimplE Embedding for Link Prediction in Knowledge Graphs (2018, background): “During the past two decades, several knowledge graphs (KGs) containing (perhaps probabilistic) facts about the world have been constructed (Carlson et al., 2010; Fader et al., 2011; Wu et al., 2012; Dong et al., 2014; Shin et al., 2015).”
Knowledge Graphs 2021: A Data Odyssey (2021, background): “Consistency constraints are a key asset for knowledge acquisition from noisy sources with joint inference over multiple candidate statements (e.g., [21, 27, 35], and indispensible for KB cleaning by eliminating false positives.”