DeepDive: Declarative Knowledge Base Construction

By Christopher De Sa, Alexander J. Ratner, Christopher R'e, Jaeho Shin, Feiran Wang, Sen Wu, Ce Zhang. 2016

155 citations recorded by the source corpus.

The dark data extraction or knowledge base construction (KBC) problem is to populate a SQL database with information from unstructured data sources including emails, webpages, and pdf reports. KBC is a long-standing problem in industry and research that encompasses problems of data extraction, cleaning, and integration. We describe DeepDive, a system that combines database and machine learning ideas to help develop KBC systems. The key idea in DeepDive is that statistical inference and machine learning are key tools to attack classical data problems in extraction, cleaning, and integration in a unified and more effective manner. DeepDive programs are declarative in that one cannot write probabilistic inference algorithms; instead, one interacts by defining features or rules about the domain. A key reason for this design choice is to enable domain experts to build their own KBC systems. We present the applications, abstractions, and techniques of DeepDive employed to accelerate construction of KBC systems.

Subfields: Computer Networks and Communications, Information Systems, Management Science and Operations Research, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

In Management Science and Operations Research

In Information Retrieval & Search

Citations in the corpus

Snorkel: Rapid Training Data Creation with Weak Supervision (2017, background): “Previous ML systems that we and others developed [52] required extensive feature engineering and model specification, leading to confusion about where to inject relevant domain knowledge.”
Seeping Semantics: Linking Datasets Using Word Embeddings for Data Discovery (2018, background): “On the other hand, knowledge construction tools [2] establish knowledge bases that link concepts and individuals via various types of semantic relationships that can”
Incremental Knowledge Base Construction Using DeepDive (2015, background): “This paper also contains content from the paper “DeepDive: Declarative Knowledge Base Construction” [62] published in the SIGMOD Record 2015.”
Snorkel: rapid training data creation with weak supervision (2019, background): “Other forms include crowdsourced labels [41,63], rules and heuristics for labeling data [47,65], and others [33,34,34,55,64].”