Probabilistic programming in Python using PyMC3

By John Salvatier, Thomas V. Wiecki, Christopher Fonnesbeck. 2016

2,362 citations recorded by the source corpus.

Probabilistic Programming allows for automatic Bayesian inference on user-defined probabilistic models. Recent advances in Markov chain Monte Carlo (MCMC) sampling allow inference on increasingly complex models. This class of MCMC, known as Hamiltonian Monte Carlo, requires gradient information which is often not readily available. PyMC3 is a new open source Probabilistic Programming framework written in Python that uses Theano to compute gradients via automatic dierentiation as well as compile probabilistic programs on-the-fly to C for increased speed. Contrary to other Probabilistic Programming languages, PyMC3 allows model specification directly in Python code. The lack of a domain specific language allows for great flexibility and direct interaction with the model. This paper is a tutorial-style introduction to this software package.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Foundation Posteriors for Approximate Probabilistic Inference (2022, neutral): “Many probabilistic programming languages (PPLs) have been developed [27, 45, 28, 44, 18, 58, 47, 53, 59, 12, 23, 17, 7, 57] each with inference approaches with strengths and weaknesses.”
Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro (2019, methodology): “Within the Python community, some examples of embedded PPLs include Pyro [2] and ProbTorch [3] based on PyTorch [4], TensorFlow Probability [5] and Edward2 [6] based on TensorFlow, and PyMC3 [7] based on Theano.”
An Introduction to Probabilistic Programming (2018, neutral): “The same is true for systems like Stan (Stan Development Team, 2014) and PyMC3 (Salvatier et al., 2016), which opt to provide users with a relatively expressive modeling language that includes if expressions, loops, and even recursion.”
Theano: A Python framework for fast computation of mathematical expressions (2016, methodology): “Another example is PyMC3 [12], a probabilistic programming framework that uses Theano to derive expressions for gradients automatically, and to generate C code for fast execution.”
Counterfactual Normalization: Proactively Addressing Dataset Shift and Improving Reliability Using Causal Mechanisms (2018, methodology): “We compute point estimates for the parameters using MAP estimation and the FITC sparse GP (Snel-son and Ghahramani, 2006) implementation in PyMC3 (Salvatier et al., 2016).”
Discriminative k-shot learning using probabilistic models (2017, methodology): “Table 2 shows that our probabilistic models are competitive with the current state-of-the-art Prototypical Networks on miniImageNet splits from [11] and outperform many of the other methods.”
Automatic differentiation in machine learning: a survey (2015, methodology): “PyMC3 (Salvatier et al., 2016) allows fitting of Bayesian models using MCMC and VI, for which it uses gradients supplied by Theano.”
Automatic Differentiation Variational Inference (2016, methodology): “The first is in PyMC3 (Salvatier et al., 2016), a probabilistic programming package, that implements advi in Python using Theano.”
Probabilistic programming with programmable inference (2018, methodology): “After inference metaprogramming was introduced in Venture, other probabilistic languages, e.g. Turing [14] and PyMC3 [42], have added limited support for custom inference in terms of subproblems.”
The JASP guidelines for conducting and reporting a Bayesian analysis (2019, methodology): “For Python, a popular package for Bayesian analyses is the PyMC3 package (Salvatier et al., 2016).”
ArviZ a unified library for exploratory analysis of Bayesian models in Python (2019, background): “ArviZ aims to be a package that integrates seamlessly with established probabilistic programming languages like PyStan (“Stan,” n.d.), PyMC (Salvatier, Wiecki, & Fonnesbeck, 2016), Edward (Tran et al., 2017, 2016), emcee (Foreman-Mackey, Hogg, Lang, & Goodman, 2013), Pyro (Bingham et al., 2018), and”
ZhuSuan: A Library for Bayesian Deep Learning (2017, methodology): “Other libraries such as PyMC3 (Salvatier et al., 2016) and Edward (Tran et al., 2016) address this problem by directly manipulating the created computation graph.”
Sensor data quality: a systematic review (2020, methodology): “6, the Bayesian estimation is carried out using the Python module PyMC [105], which is designed to implement Bayesian statistical models.”
Gen: a general-purpose probabilistic programming system with programmable inference (2019, background): “Researchers have introduced many probabilistic programming systems over the last 20 years [8, 18, 22, 23, 40–42, 48, 49, 56, 61, 68].”