TensorFlow Distributions

By Joshua V. Dillon, Ian Langmore, Dustin Tran, Eugene Brevdo, Srinivas Vasudevan, David A. Moore, Brian Patton, Alexander A. Alemi, Matthew D. Hoffman, Rif A. Saurous. 2017

401 citations recorded by the source corpus.

The TensorFlow Distributions library implements a vision of probability theory adapted to the modern deep-learning paradigm of end-to-end differentiable computation. Building on two basic abstractions, it offers flexible building blocks for probabilistic computation. Distributions provide fast, numerically stable methods for generating samples and computing statistics, e.g., log density. Bijectors provide composable volume-tracking transformations with automatic caching. Together these enable modular construction of high dimensional distributions and transformations not possible with previous libraries (e.g., pixelCNNs, autoregressive flows, and reversible residual networks). They are the workhorse behind deep probabilistic programming systems like Edward and empower fast black-box inference in probabilistic models built on deep-network components. TensorFlow Distributions has proven an important part of the TensorFlow toolkit within Google and in the broader deep learning community.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Probabilistic & Bayesian Methods

Citations in the corpus

Pyro: Deep Universal Probabilistic Programming (2018, background): “s. As a result, several of the authors made substantial open-source contributions 3 Bingham et al. upstream to PyTorch Distributions,1 a new PyTorch core library inspired by TensorFlow Distributions (Dillon et al. (2017)). 4. Existing Systems System Expressivity: Dynamic control Scalability: Subsamp”
Neural Spline Flows (2019, neutral): “A third-party implementation has also been added to TensorFlow Probability [8].”
Resampled Priors for Variational Autoencoders (2018, methodology): “We quantitatively evaluate all methods using the negative test log likelihood (NLL) estimated using 1000 importance samples [3].”
Simple, Distributed, and Accelerated Probabilistic Programming (2018, methodology): “Random variables provide methods such as log_prob and sample , wrapping TensorFlow Distributions [10].”
Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale (2019, neutral): “Modern PPLs such as Pyro [11] and TensorFlow Probability [19, 70] use gradient-based inference engines including variational inference [36, 42] and Hamiltonian Monte Carlo [37, 57] that benefit from modern deep learning hardware and automatic differentiation [9] features provided by PyTorch [61]…”
Variational Federated Multi-Task Learning (2019, methodology): “Implementation of VIRTUAL is based on tensorflow [1] and tensorflow distributions [13] packages.”
WAIC, but Why? Generative Ensembles for Robust Anomaly Detection (2018, methodology): “We extend the VAE example code2 from Tensorflow Probability (Dillon et al., 2017) to use a Masked Autoregressive Flow prior (Papamakarios et al., 2017), and train the model for 5k steps: The encoder consists of 5 convoluton layers.”
Bayesian Layers: A Module for Neural Network Uncertainty (2018, background): “A natural idea is to design a new abstraction for invertible functions (Dillon et al., 2017).”
Hierarchical Indian buffet neural networks for Bayesian continual learning (2019, methodology): “Implicit reparameterisation of the Beta distribution is readily implemented in TensorFlow Distributions [Dillon et al., 2017].”
Effect Handling for Composable Program Transformations in Edward2 (2018, methodology): “As an example application requiring nested interceptors, we implement an unconstraining interceptor, using the Bi-jectors library to handle Jacobian corrections [2]: Here the ed.”
Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model (2018, methodology): “[32] J.”
Importance Weighted Hierarchical Variational Inference (2019, methodology): “For flow-based models we used their implementations from TensorFlow Probability (Dillon et al., 2017).”
Pathfinder: Parallel quasi-Newton variational inference (2021, neutral): “…variational inference algorithm, implemented in Stan (Stan Development Team, 2021a), PyMC3 (Salvatier et al., 2016), Pyro (Bingham et al., 2019), TensorFlow Probability (Dillon et al., 2017), JAX (Bradbury et al., 2018), Turing.jl (Ge et al., 2018), and other differentiable programming languages.”