Structured Priors for Structure Learning

By Vikash K. Mansinghka, Charles Kemp, T. Griffiths, Josh Tenenbaum. 2006

85 citations recorded by the source corpus.

Traditional approaches to Bayes net structure learning typically assume little regularity in graph structure other than sparseness. However, in many cases, we expect more systematicity: variables in real-world systems often group into classes that predict the kinds of probabilistic dependencies they participate in. Here we capture this form of prior knowledge in a hierarchical Bayesian framework, and exploit it to enable structure learning and type discovery from small datasets. Specifically, we present a nonparametric generative model for directed acyclic graphs as a prior for Bayes net structure learning. Our model assumes that variables come in one or more classes and that the prior probability of an edge existing between two variables is a function only of their classes. We derive an MCMC algorithm for simultaneous inference of the number of classes, the class assignments of variables, and the Bayes net structure over variables. For several realistic, sparse datasets, we show that the bias towards systematicity of connections provided by our model yields more accurate learned networks than a traditional, uniform prior approach, and that the classes found by our model are appropriate.

Subfields: Artificial Intelligence, Molecular Biology, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Probabilistic & Bayesian Methods

Citations in the corpus

Bayesian Structure Learning with Generative Flow Networks (2022, methodology): “This was further reﬁned by either modifying the underlying space of the Markov chain (Kuipers and Moffa, 2017; Niinimäki et al., 2016), or its local moves (Mansinghka et al., 2006; Eaton and Murphy, 2007a; Kuipers et al., 2021).”
Venture: a higher-order probabilistic programming platform with programmable inference (2014, background): “, 2007; Rasmussen and Williams, 2006; Griffiths and Ghahramani, 2005), probabilistic models over worlds with a priori unknown numbers of objects, models for learning the structure of graphical models (Heckerman, 1998; Friedman and Koller, 2003; Mansinghka et al., 2006), models for inductive learning”
Learning overhypotheses with hierarchical Bayesian models. (2007, background): “We defined models that generate scalars (α) and vectors (β β β β, θ θ θ θ, y), but hierarchical probabilistic models can generate many other kinds of representations, including taxonomies (Kemp, Perfors & Tenenbaum, 2004), ontologies (Schmidt, Kemp & Tenenbaum, 2006), causal networks ( Mansinghka, K”
Sparse and Locally Constant Gaussian Graphical Models (2009, background): “silhouettes) and that variables far apart are only weakly correlated [8], interaction between a priori known groups of variables as in [9], or block structures as in [10] in the context of Bayesian networks.”
Exact Bayesian structure learning from uncertain interventions (2007, background): “(See [MKTG06] for some ideas on how to partition nodes into groups/ layers in an unsupervised way.”
Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons (2011, background): “The ASIA-network is an example for a larger class of Bayesian networks that are of special interest from the perspective of Cognitive Science [25].”
Learning Bayesian networks: approaches and issues (2011, unknown): “Mansinghka et al. (2006) also look at non-expert supplied priors.”
Applications of Probabilistic Programming (Master's thesis, 2015) (2016, methodology): “Our approach is also related to probabilistic model learning, for example to learning probabilistic relational models (Friedman et al., 1999) and Bayesian network structure (Mansinghka et al., 2012).”
Bayesian structure learning using dynamic programming and MCMC (2007, methodology): “;swindow[MR94].) Unfortunately, this proposal is very local, and the resulting chains do not mix well in more than about 10 dimensions. An alternative is to use Gibbs sampling on the adjacency matrix [MKTG06]. In our experience, this gets stuck even more easily, althoughthiscanbeamelioratedsomewhatb”
Bayesian models of cognition (2008, methodology): “…graphical models, using probabilistic forms of generative grammars, predicate logic, or other structured representations (Griﬃths, 2005; Griﬃths & Tenenbaum, 2007b; Mansinghka, Kemp, Tenenbaum, & Griﬃths, 2006; Tenenbaum et al., 2006; Tenenbaum, Griﬃths, & Niyogi, 2007; Tenenbaum & Niyogi, 2003).”