Posterior Regularization for Structured Latent Variable Models

By Kuzman Ganchev, João Graça, Jennifer Gillenwater, B. Taskar. 2010

573 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Probabilistic & Bayesian Methods

Citations in the corpus

Joint Concept Learning and Semantic Parsing from Natural Language Explanations (2017, methodology): “Notable approaches such as Generalized Expectation (Mann and McCallum, 2010) and Posterior Regularization (Ganchev et al., 2010) have explored integration of manually provided ‘side-information’ (feature and label constraints) to guide machine learning models.”
Benefits of Intermediate Annotations in Reading Comprehension (2020, background): “Our auxiliary training with intermediate annotations is inspired by extensive related work on training models using side information or domain knowledge beyond labels (Mann and McCallum, 2008; Chang et al., 2007; Ganchev et al., 2010; Rock-taschel et al., 2015).”
Counterfactual Fairness (2017, background): “As U and A are not independent given X in this representation, a type of “posterior regularization” (Ganchev et al., 2010) is enforced such that a posterior pfair(U |A,X) is close to the model posterior p(U | A,X) while satisfying pfair(U |A = a,X) ≈ pfair(U |A = a′, X).”
Injecting Logical Background Knowledge into Embeddings for Relation Extraction (2015, methodology): “Weakly Supervised Learning Our work is also inspired by weakly supervised approaches (Ganchev et al., 2010) that use structural constraints as a source of indirect supervision, and have been used for several NLP tasks (Chang et al.”
Bayesian representation learning with oracle constraints (2015, positive): “Using side-knowledge as a regularizer for the posterior over latent variables has been explored in other settings for simpler latent variable models in Ganchev et al. (2010) and we take inspiration from that work.”
Constrained Convolutional Neural Networks for Weakly Supervised Segmentation (2015, methodology): “[12] constrain an auxiliary variable yielding an algorithm similar to our objective in dual space.”
Good-Enough Compositional Data Augmentation (2019, methodology): “…tools available for improving the function approximators or loss functions themselves—regularization (Oh et al., 2017), posterior regularization (Ganchev et al., 2010; Hu et al., 2018), explicit stacks (Grefenstette et al., 2015) and composition operators (Bowman et al., 2016); these existing…”
Label-Free Supervision of Neural Networks with Physics and Domain Knowledge (2016, neutral): “Natural language processing has seen many successful applications of constraint learning (Liang, Jordan, and Klein 2009; Chang, Ratinov, and Roth 2007; Ganchev et al. 2010), and the recent work of (Ratner et al.”
Mapping to Declarative Knowledge for Word Problem Solving (2017, background): “…recent approaches tried to incorporate knowledge in the form of constraints or expectations from the output (Roth and tau Yih, 2004; wei Chang et al., 2007; Chang et al., 2012; Ganchev et al., 2010; Smith and Eisner, 2006; Naseem et al., 2010; Bisk and Hockenmaier, 2012; Gimpel and Bansal, 2014).”
Logic-Guided Data Augmentation and Regularization for Consistent Question Answering (2020, neutral): “Regularizing loss to penalize violations of structural constraints in models’ output has been also studied in previous work on constraint satisfaction in structured learning (Lee et al., 2019; Ganchev et al., 2010).”
Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints (2018, methodology): “Our approach, which incorporates distributional constraints into the decoding objective, is related to prior work on posterior regularization (Mann and McCallum, 2008; Ganchev et al., 2010; Zhu et al., 2014).”
Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence (2020, methodology): “To control the granularity as we want, we utilize posterior regularization (Ganchev et al., 2010) to constrain the expected number of segments for each text 5 , which can be calculated by going through a similar forward pass as in Equation 4 (Eisner, 2002; Backes et al., 2018).”
Learning Semantic Representations for the Phrase Translation Model (2013, methodology): “BLTM with Posterior Regularization (BLTMPR) is trained on parallel training data using the EM algorithm with a constraint enforcing a source sentence and its paralleled target sentence to not only share the same prior topic distribution, but to also have similar fractions of words assigned to each t”
Learning Syntactic Categories Using Paradigmatic Representations of Word Context (2012, methodology): “562 (Ganchev et al., 2010)* .”
Weakly Supervised Object Detection with Posterior Regularization (2014, methodology): “[11] introduce a probabilistic framework that can express and enforce data-dependent constraints on latent variable distribution with posterior regularization.”
Be Consistent! Improving Procedural Text Comprehension using Label Consistency (2019, background): “In particular, it encourages label consistency during end-to-end training of a neural model, allowing consistency bias to improve the model itself, rather than be enforced in a post-processing step, e.g., posterior regularization (Ganchev et al., 2010).”
MAE: Mutual Posterior-Divergence Regularization for Variational AutoEncoders (2019, background): “It may instead focus on explaining irrelevant but common correlations in the data (Ganchev et al., 2010).”