Hinge-Loss Markov Random Fields and Probabilistic Soft Logic

By Stephen H. Bach, Matthias Broecheler, Bert Huang, Lise Getoor. 2015

423 citations recorded by the source corpus.

A fundamental challenge in developing high-impact machine learning technologies is balancing the need to model rich, structured domains with the ability to scale to big data. Many important problem areas are both richly structured and large scale, from social and biological networks, to knowledge graphs and the Web, to images, video, and natural language. In this paper, we introduce two new formalisms for modeling structured data, and show that they can both capture rich structure and scale to big data. The first, hinge-loss Markov random fields (HL-MRFs), is a new kind of probabilistic graphical model that generalizes different approaches to convex inference. We unite three approaches from the randomized algorithms, probabilistic graphical models, and fuzzy logic communities, showing that all three lead to the same inference objective. We then define HL-MRFs by generalizing this unified objective. The second new formalism, probabilistic soft logic (PSL), is a probabilistic programming language that makes HL-MRFs easy to define using a syntax based on first-order logic. We introduce an algorithm for inferring most-probable variable assignments (MAP inference) that is much more scalable than general-purpose convex optimization methods, because it uses message passing to take advantage of sparse dependency structures. We then show how to learn the parameters of HL-MRFs. The learned HL-MRFs are as accurate as analogous discrete models, but much more scalable. Together, these algorithms enable HL-MRFs and PSL to model rich, structured data at scales not previously possible.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Integrating Learning and Reasoning with Deep Logic Models (2019, methodology): “However in PSL, rule weights are restricted to only positive values denoting how far the rule is from being satisﬁed.”
Logic Tensor Networks for Semantic Image Interpretation (2017, methodology): “In [2], a probabilistic fuzzy logic is used, but not with real semantics.”
A Review of Relational Machine Learning for Knowledge Graphs (2015, background): “If one restricts the class of potential functions to be just disjunctions (using or and not, but no and), then one obtains a (special case of) hinge loss MRF (HL-MRFs) [130], for which efficient convex algorithms can be applied, based on a continuous relaxation of the binary random variables.”
HoloClean: Holistic Data Repairs with Probabilistic Inference (2017, background): “Recently, declarative probabilistic frameworks [2, 6, 46] have been introduced to facilitate the construction of large scale factor graphs.”
NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language (2019, methodology): “Our work is very related to the integration of Markov Logic Networks (Richardson and Domingos, 2006) and Probabilistic Soft Logic (Bach et al., 2017) with word embeddings, which was applied to Recognizing Textual Entailment (RTE) and Semantic Textual Similarity (STS) tasks (Garrette et al., 2011, 20”
Fairness in Relational Domains (2018, methodology): “PSL is a probabilistic programming language for defining hingeloss Markov random fields [5].”
Domain-specific Knowledge Extraction from the Web of Data (2018, methodology): “PSL uses lukasiewicz t-norm [Bach et al. 2015] to provide a relaxation for the logical connectives ∧ and ∨.”
Probabilistic logic programs: Unifying program trace and possible world semantics (2017, background): “Furthermore, as extensions of Prolog, probabilistic logic programming languages are Turing equivalent, a property that they share with probabilistic programming languages extending traditional functional or imperative programming languages, and which distinguishes them from many of the statistical r”
From LSAT: The Progress and Challenges of Complex Reasoning (2021, methodology): “Advanced methods to solve complex reasoning problems, no matter what specific reasoning skill is required, can be summarized as following three types: symbolic models, neural models and neural-symbolic models [15, 16, 17].”
Domain-Specific Use Cases for Knowledge-Enabled Social Media Analysis (2018, methodology): “The approach combines the semantic type, lexical, and structural categories using probabilistic soft logic (PSL) [2] to incorporate different forms of evidence.”
Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning (2023, background): “…example, statistical relational learning (SRL) systems such as Markov logic networks (Richardson and Domingos, 2006) and probabilistic soft logic (Bach et al., 2017) have demonstrated great promise in reasoning under uncertainty and integration with our framework would enable even more adaptive…”
On a Convex Logic Fragment for Learning and Reasoning (2018, background): “cal constraints and like other authors do [22], [24], we decided to focus on Łukasiewicz logic.”
Explicit Reasoning over End-to-End Neural Architectures for Visual Question Answering (2018, methodology): “To complement the current successful end-to-end systems, we developed two major add-on components: 1) a semantic parsing module for questions and captions, and 2) an augmented reasoning engine based on Probabilistic Soft Logic (PSL) (Bach et al. 2015).”
DualDiv: diversifying items and explanation styles in explainable hybrid recommendation (2019, methodology): “HyPER makes use of probabilistic soft logic (PSL) [2] that enables HyPER to develop recommendation models through a set of template rules in first-order logic syntax.”
SLiMFast: Guaranteed Results for Data Fusion and Source Reliability (2015, methodology): “Any declarative factor graph framework (e.g., Alchemy [1], DeepDive [2, 35], PSL [5]) can by used.”
OptTyper: Probabilistic Type Inference by Optimising Logical and Natural Constraints (2020, background): “A key idea is that there are two sorts of constraints in step (3): logical constraints and natural constraints.”
User Preferences for Hybrid Explanations (2017, methodology): “Here, we use HyPER [14] which makes use of probabilistic soft logic (PSL) [1], a generic probabilistic reasoning framework.”