Church: a language for generative models

By Noah D. Goodman, Vikash K. Mansinghka, Daniel M. Roy, Keith Bonawitz, Josh Tenenbaum. 2008

843 citations recorded by the source corpus.

Formal languages for probabilistic modeling enable re-use, modularity, and descriptive clarity, and can foster generic inference techniques. We introduce Church, a universal language for describing stochastic generative processes. Church is based on the Lisp model of lambda calculus, containing a pure Lisp as its deterministic subset. The semantics of Church is defined in terms of evaluation histories and conditional distributions on such histories. Church also includes a novel language construct, the stochastic memoizer, which enables simple description of many complex non-parametric models. We illustrate language features through several examples, including: a generalized Bayes net in which parameters cluster over trials, infinite PCFGs, planning by inference, and various non-parametric clustering models. Finally, we show how to implement query on any Church program, exactly and approximately, using Monte Carlo techniques.

Subfields: Artificial Intelligence, Computer Networks and Communications, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Probabilistic & Bayesian Methods

Citations in the corpus

Foundation Posteriors for Approximate Probabilistic Inference (2022, neutral): “Many probabilistic programming languages (PPLs) have been developed [27, 45, 28, 44, 18, 58, 47, 53, 59, 12, 23, 17, 7, 57] each with inference approaches with strengths and weaknesses.”
An Introduction to Probabilistic Programming (2018, background): “, 2016); machine learning: Church (Goodman et al., 2008), Anglican (Wood et al.”
Design and Implementation of Probabilistic Programming Language Anglican (2016, methodology): “However, there are also languages which follow imperative paradigm and support mutable state directly [5, 23], or through probabilistic primitives [6, 9].”
Relational inductive biases, deep learning, and graph networks (2018, background): “The question of how to build artificial systems which exhibit combinatorial generalization has been at the heart of AI since its origins, and was central to many structured approaches, including logic, grammars, classic planning, graphical models, causal reasoning, Bayesian nonparametrics, and proba”
Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages (2015, background): “This is a goal ∗Currently at Google DeepMind. sought, for instance, in probabilistic programming languages (Wingate et al., 2011; Goodman et al., 2008), as well as in more granular, component-based systems (Stan Development Team, 2014; Minka et al., 2014).”
Learning Disentangled Representations with Semi-Supervised Deep Generative Models (2017, methodology): “ich encode the requisite structure and computation. This provides another direction to explore in the future — the extension of the stochastic computation graph framework to probabilistic programming [9, 34, 35]. Probabilistic programs go beyond the presented framework to permit more expressive mode”
Programming with a Differentiable Forth Interpreter (2016, background): “Probabilistic and Bayesian Programming Our work is closely related to probabilistic programming languages such as Church (Goodman et al., 2008).”
Probabilistic programming in Python using PyMC3 (2016, background): “Each of these systems are domain-speciﬁc languages built on top of existing low-level languages; notable examples include Church (Goodman et al., 2012) (derived from Scheme), Anglican (Wood et al., 2014) (integrated with Clojure and compiled with a Java Virtual Machine), Venture (Mansinghka et al., ”
Automated Variational Inference in Probabilistic Programming (2013, methodology): “Examples include BLOG [1], PRISM [2], Bayesian Logic Programs [3], Stochastic Logic Programs [4], Independent Choice Logic [5], IBAL [6], Probabilistic Scheme [7], Λ ◦ [8], Church [9], Stochastic Matlab [10], and HANSEI [11].”
Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs (2013, background): “We then run the kernel associated with this variable, and accept or reject via the Metropolis-Hastings equation: αMH((S,X)→ (S′, X ′)) = min ( 1, P (ID|f(S′, X ′), X ′)P (S′)P (X ′)q((S′, X ′)→ (S,X)) P (ID|f(S,X), X)P (S)P (X)q((S,X)→ (S′, X ′)) ) We implement our probabilistic graphics programs in”
TerpreT: A Probabilistic Programming Language for Program Induction (2016, background): “istic programming systems specialized to diﬀerent use-cases. One dominant axis of variability is in the expressiveness of the language. Some probabilistic programming languages, exempliﬁed by Church (Goodman et al., 2008), allow great freedom in the expressibility of the language, including construc”
A New Approach to Probabilistic Programming Inference (2014, methodology): “Related approaches include rejection sampling [3], singlesite MH [3, 12].”
Edward: A library for probabilistic modeling, inference, and criticism (2016, methodology): “Recent works include Church (Goodman et al., 2012), Venture (Mansinghka et al.”
Inference Compilation and Universal Probabilistic Programming (2016, methodology): “We call what we do “compilation of inference” because our method transforms a denotational specification of an inference problem in the form of a probabilistic program written in a universal programming language into a trained neural network denoted in a neural network specification language.”
Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints (2016, neutral): “Some languages (such as Infer.net [21], PyMC [26], and Stan [33]) are less expressive but provide powerful inference algorithms, while others (such as Anglican [34], Church [12], and Venture [20]) have less efﬁcient inference algorithms but more expressive power.”