Venture: a higher-order probabilistic programming platform with programmable inference

By Vikash K. Mansinghka, Daniel Selsam, Yura N. Perov. 2014

262 citations recorded by the source corpus.

We describe Venture, an interactive virtual machine for probabilistic programming that aims to be sufficiently expressive, extensible, and efficient for general-purpose use. Like Church, probabilistic models and inference problems in Venture are specified via a Turing-complete, higher-order probabilistic language descended from Lisp. Unlike Church, Venture also provides a compositional language for custom inference strategies built out of scalable exact and approximate techniques. We also describe four key aspects of Venture's implementation that build on ideas from probabilistic graphical models. First, we describe the stochastic procedure interface (SPI) that specifies and encapsulates primitive random variables. The SPI supports custom control flow, higher-order probabilistic procedures, partially exchangeable sequences and ``likelihood-free'' stochastic simulators. It also supports external models that do inference over latent variables hidden from Venture. Second, we describe probabilistic execution traces (PETs), which represent execution histories of Venture programs. PETs capture conditional dependencies, existential dependencies and exchangeable coupling. Third, we describe partitions of execution histories called scaffolds that factor global inference problems into coherent sub-problems. Finally, we describe a family of stochastic regeneration algorithms for efficiently modifying PET fragments contained within scaffolds. Stochastic regeneration linear runtime scaling in cases where many previous approaches scaled quadratically. We show how to use stochastic regeneration and the SPI to implement general-purpose inference strategies such as Metropolis-Hastings, Gibbs sampling, and blocked proposals based on particle Markov chain Monte Carlo and mean-field variational inference techniques.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Foundation Posteriors for Approximate Probabilistic Inference (2022, neutral): “Many probabilistic programming languages (PPLs) have been developed [27, 45, 28, 44, 18, 58, 47, 53, 59, 12, 23, 17, 7, 57] each with inference approaches with strengths and weaknesses.”
An Introduction to Probabilistic Programming (2018, neutral): “, 2015), Venture (Mansinghka et al., 2014), Probabilistic-C (Paige and Wood, 2014), webPPL (Goodman and Stuhlmüller, 2014), CPProb (Casado, 2017), (Koller et al.”
Design and Implementation of Probabilistic Programming Language Anglican (2016, methodology): “Probabilistic programming systems (Goodman et al. 2008; Mansinghka et al. 2014; Wood et al. 2014; Goodman and Stuhlmüller 2015) represent generative models as programs written in a specialized language that provides syntax for the definition and conditioning of random variables.”
Picture: A probabilistic programming language for scene perception (2015, neutral): “…of random variables as in equation 1) can then be accepted with probability: min 1 , P (cid:15) ( ν ( I D ) − ν ( I (cid:48) R )) P ( S (cid:48) ρ ) P ( X (cid:48) ρ ) q ( S (cid:48) ,X (cid:48) ) ( S,X ) P (cid:15) ( ν ( I D ) − ν ( I R )) P ( S ρ ) P ( X ρ ) q ( S,X ) ( S (cid:48) ,X (cid:48) ) .”
Probabilistic programming in Python using PyMC3 (2016, background): “…low-level languages; notable examples include Church (Goodman et al., 2012) (derived from Scheme), Anglican (Wood et al., 2014) (integrated with Clojure and compiled with a Java Virtual Machine), Venture (Mansinghka et al., 2014) (built from C++), Infer.NET (Minka et al., 2010) (built upon the .”
Automatic Differentiation Variational Inference (2016, background): “, 2008), Figaro (Pfeffer, 2009), Venture (Mansinghka et al., 2014), and Anglican (Wood et al.”
Inference Compilation and Universal Probabilistic Programming (2016, methodology): “Probabilistic programming systems such as STAN (Car-penter et al., 2015), BUGS (Lunn et al., 2000), and Infer.NET (Minka et al., 2014) allow eﬃcient inference in a restricted space of generative models, while systems such as Church (Goodman et al., 2008), Venture (Mansinghka et al., 2014), and Angli”
Commutative Semantics for Probabilistic Programming (2017, background): “Our language syntax is not novel: it is the same language as in [43], and as such an idealized, typed, first order version of Anglican [46], Church [11], Hakaru [30], Venture [26] and so on.”
Automatic model construction with Gaussian processes (2014, background): “From the top-down, the probabilistic programming community is developing automatic inference engines for extremely broad classes of models (Goodman et al., 2008; Koller et al., 1997; Mansinghka et al., 2014; Milch et al., 2007; Stan Development Team, 2014; Wood et al., 2014) such as the class of all”
Meta-Learning an Inference Algorithm for Probabilistic Programs (2021, background): “This line of research has led to performant probabilistic programming systems (Goodman et al., 2008; Wood et al., 2014; Mansinghka et al., 2014; Minka et al., 2018; Narayanan et al., 2016; Salvatier et al., 2016; Carpenter et al., 2017; Tran et al., 2016; Ge et al., 2018; Bingham et al., 2018).”
Output-Sensitive Adaptive Metropolis-Hastings for Probabilistic Programs (2015, methodology): “Our primary interest in this work is to improve MCMC methods for probabilistic programming [6,7,11,16].”
Inverse Graphics with Probabilistic CAD Models (2014, background): “Many of these directions, as well as the exploration of alternative inference strategies, may be simpliﬁed by implementation as generative probabilistic graphics programs atop general-purpose probabilistic programming systems [10].”
A lambda-calculus foundation for universal probabilistic programming (2015, background): “CHURCH, and its descendants VENTURE (Mansinghka et al. 2014), ANGLICAN (Tolpin et al. 2015), and WEB CHURCH (Goodman and Tenenbaum 2014) are dialects of SCHEME.”
Deep Amortized Inference for Probabilistic Programs (2016, unknown): “A universal PPL, one built on a Turing-complete language, can represent any computable probability distribution, including open-world models, Bayesian non-parameterics, and stochastic recursion [6, 19, 33].”
MultiVerse: Causal Reasoning using Importance Sampling in Probabilistic Programming (2019, methodology): “Also, similar ideas can be implemented in existing probabilistic programming languages like Pyro, Anglican (Wood et al., 2014; Tolpin et al., 2015, 2016), Venture, Church (Goodman et al., 2012) language engines, Gen (Cusumano-Towner and Mansinghka, 2018), and others (Probabilistic Programming Wiki A”
Probabilistic programming with programmable inference (2018, methodology): “…better support this increasingly prominent class of computations, researchers have designed and implemented probabilistic programming languages, which provide direct support for defining probabilistic models and performing probabilistic inference within the language [3, 16– 19, 23, 28, 31, 47, 50].”
Biips: Software for Bayesian Inference with Interacting Particle Systems (2014, methodology): “Other software such as Venture (Mansinghka et al. 2014) or Anglican (Wood et al. 2014) also propose particle MCMC inference engines for posterior inference, with a diﬀerent probabilistic language, that allows for more expressiveness; in particular, it can deal with models of changing dimensions, com”
PSI: Exact Symbolic Inference for Probabilistic Programs (2016, methodology): “The general approaches include sampling-based Monte Carlo methods [20,40,21,19,39,46,32] and projections to convenient probability distributions, such as variational inference [34,8] or discretization [31,12].”