Combining Generative and Discriminative Model Scores for Distant Supervision

By Benjamin Roth, Dietrich Klakow. 2013

44 citations recorded by the source corpus.

Distant supervision is a scheme to generate noisy training data for relation extraction by aligning entities of a knowledge base with text. In this work we combine the output of a discriminative at-least-one learner with that of a generative hierarchical topic model to reduce the noise in distant supervision data. The combination significantly increases the ranking quality of extracted facts and achieves state-of-the-art extraction performance in an end-to-end setting. A simple linear interpolation of the model scores performs better than a parameter-free scheme based on nondominated sorting.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

Citations in the corpus

Snorkel: Rapid Training Data Creation with Weak Supervision (2017, methodology): “Other methods use generative models with hand-specified dependency structures to label data for specific modalities, such as topic models for text [4] or denoising distant supervision sources [42, 48].”
Training Classifiers with Natural Language Explanations (2018, background): “Various extensions have focused on aggregating a variety of supervision sources by learning generative models from noisy labels (Alfonseca et al., 2012; Takamatsu et al., 2012; Roth and Klakow, 2013; Ratner et al., 2016; Varma et al., 2017).”
Data Programming: Creating Large Training Sets, Quickly (2016, methodology): “Other extensions actually model the accuracy of this pattern w using a discriminative feature-based model [25], or generative models such as hierarchical topic models [1,24,29].”
Relation Extraction Using Distant Supervision (2018, background): “The combination of hierarchical topic model and at-least-one learner is implemented in [55].”
Learning the Structure of Generative Models without Labeled Data (2017, methodology): “Others use hierarchical topic models to generate additional training data for weak supervision, but they do not support user-provided heuristics (Alfonseca et al., 2012; Takamatsu et al., 2012; Roth & Klakow, 2013a;b).”
Socratic Learning: Augmenting Generative Models to Incorporate Latent Subsets in Training Data (2016, methodology): “Recently, generative models have been used to combine various sources of weak supervision [1, 27, 23].”
Snuba: Automating Weak Supervision to Label Training Data (2018, methodology): “Other methods, including our previous work [52, 40, 54], rely on generative models to learn accuracies and dependencies among labeling sources [2, 42, 50].”
Inferring Generative Model Structure with Static Analysis (2017, background): “Recently, generative models have been used to create training labels from various weak supervision sources, such as heuristics or knowledge bases, by modeling the true class label as a latent variable [1, 2, 27, 31, 36, 37].”
Snorkel: rapid training data creation with weak supervision (2019, methodology): “Other methods use generative models with hand-specified dependency structures to label data for specific modalities, such as topic models for text [3] or denoising distant supervision sources [50,57].”