Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation

By David Wingate, Andreas Stuhlmüller, Noah D. Goodman. 2011

196 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods, Program Analysis & Verification

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

In Program Analysis & Verification

Citations in the corpus

Pyro: Deep Universal Probabilistic Programming (2018, background): “Anglican (Tolpin et al. (2016)) and webPPL (Goodman and Stuhlm¨uller (2014)) are lightweight successors to Church, embedded as syntactic subsets (Wingate et al. (2011)) in the general-purpose programming languages Clojure and JavaScript.”
Design and Implementation of Probabilistic Programming Language Anglican (2016, methodology): “Among the available inference algorithms there are Particle Cascade (Paige et al. 2014), Lightweight Metropolis-Hastings (Wingate et al. 2011), Iterative Conditional Sequential Monte-Carlo (Particle Gibbs) (Wood et al. 2014), and others.”
Learning Disentangled Representations with Semi-Supervised Deep Generative Models (2017, background): “This provides another direction to explore in the future — the extension of the stochastic computation graph framework to probabilistic programming [9, 34, 35].”
Picture: A probabilistic programming language for scene perception (2015, methodology): “We use the transformational compilation technique [46] to implement Picture , which is a general method of transforming arbitrary programming languages into probabilistic programming languages.”
Automated Variational Inference in Probabilistic Programming (2013, methodology): “Examples include BLOG [1], PRISM [2], Bayesian Logic Programs [3], Stochastic Logic Programs [4], Independent Choice Logic [5], IBAL [6], Probabilistic Scheme [7], Λ ◦ [8], Church [9], Stochastic Matlab [10], and HANSEI [11].”
A New Approach to Probabilistic Programming Inference (2014, methodology): “Related approaches include rejection sampling [3], singlesite MH [3, 12].”
Venture: a higher-order probabilistic programming platform with programmable inference (2014, methodology): “For “lightweight” implementations based on direct transformational compilation or augmented interpretation, such as Bher (Wingate et al., 2011b), this is a basic consequence of the approach: the entire program must be re-simulated to make a change to a single latent variable.”
Inference Compilation and Universal Probabilistic Programming (2016, methodology): “While there are many inference algorithms for universal probabilistic programming languages (Wingate et al., 2011; Ritchie et al., 2016a; Wood et al., 2014; Paige et al., 2014; Rainforth et al., 2016), we focus on algorithms in the importance sampling family in the context of which we will develop…”
Denotational validation of higher-order Bayesian inference (2017, methodology): “The early version of the lightweight Metropolis-Hastings algorithm had a bug because of an incorrect acceptance ratio [Wingate et al. 2011].”
Deep Amortized Inference for Probabilistic Programs (2016, methodology): “Our system names parameters according to where they are declared in the program execution trace, using the same naming technique as is used for random choices in probabilistic programming MCMC engines [31].”
Output-Sensitive Adaptive Metropolis-Hastings for Probabilistic Programs (2015, methodology): “LMH [15] is a sampling scheme for probabilistic programs where a single random variable drawn in the course of a particular execution of a probabilistic program is modified via a standard MH proposal, and this modification is accepted by comparing the values of the joint probability of old and new p”
A Compilation Target for Probabilistic Programming Languages (2014, methodology): “…but is an interpreted language based on Scheme, implemented in Clojure, and running on the JVM; and probabilistic-js 1 , a compiled system implementing the inference approach in (Wingate et al., 2011), which runs Metropolis-Hastings over each individual random choice in the program execution trace.”
Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model (2018, methodology): “The LMH and RMH engines we implement are specialized for sampling in the space of execution traces of probabilistic programs, and provide a way of sampling from the true posterior and therefore provide a baseline—at a high computational cost.”
Probabilistic programming (2014, result): “In contrast, previous approaches [21,49,58,59] have not specifically addressed rejections due to failing observations.”
Particle Gibbs with Ancestor Sampling for Probabilistic Programs (2015, methodology): “Another well-known inference technique for probabilistic programs is lightweight Metropolis-Hasting (LMH) [Wingate et al., 2011].”
Nonstandard Interpretations of Probabilistic Programs for Efficient Inference (2011, methodology): “5 ) 3: X(i) = randn; 4: else 5: X(i) = gammarnd; 6: end; 7: end; We begin by outlining our setup, following [28].”
A lambda-calculus foundation for universal probabilistic programming (2015, background): “In future, we aim to extend our proofs to cover more efﬁcient algorithms, inspired by Wingate et al. (2011) and Kiselyov (2016), for example.”