Doubly Stochastic Variational Bayes for non-Conjugate Inference

By Michalis K. Titsias, Miguel L'azaro-Gredilla. 2014

400 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Variational Inference with Normalizing Flows (2015, background): “This forms what has been aptly named doubly-stochastic estimation (Titsias & Lazaro-Gredilla, 2014), since we have one source of stochasticity from the mini-batch and a second from the Monte Carlo approximation of the expectation.”
The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables (2016, methodology): “(7) The reparameterization trick, introduced in the context of variational inference independently by Kingma & Welling (2014), Rezende et al. (2014), and Titsias & Lázaro-Gredilla (2014), is usually the estimator of choice when it is applicable.”
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning (2015, methodology): “Recent advances in variational inference introduced new techniques into the ﬁeld such as sampling-based variational inference and stochastic variational inference (Blei et al., 2012; Kingma & Welling, 2013; Rezende et al., 2014; Titsias & L´azaro-Gredilla, 2014; Hoffman et al., 2013).”
Variational Inference: A Review for Statisticians (2016, background): “As examples, see Kingma and Welling (2014); Rezende et al. (2014); Ranganath et al. (2014, 2016); Salimans and Knowles (2014); Titsias and Lázaro-Gredilla (2014), and Tran et al. (2016).”
Counterfactual inference for consumer choice across many product categories (2019, methodology): “We can do this by applying an approach that is known as the reparametrization trick (Kingma and Welling, 2014; Titsias and Lázaro-Gredilla, 2014; Rezende et al., 2014).”
SHOPPER: A Probabilistic Model of Consumer Choice with Substitutes and Complements (2017, methodology): “One example is language modeling and, in particular, word embedding models. the gradient (Kingma and Welling, 2014; Titsias and Lázaro-Gredilla, 2014; Rezende, Mohamed and Wierstra, 2014).”
Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference (2015, background): “The stochastic optimisation objective converges to the same limit as the full Bayesian model [20, 21, 22, 23, 24].”
Weight Uncertainty in Neural Networks (2015, background): “Titsias and L´azaro-Gredilla (2014) considered a large-scale logistic regression task.”
Concrete Dropout (2017, background): “Several estimators are available for us to do this: for example the score function estimator (also known as a likelihood ratio estimator and Reinforce [6, 12, 30, 35]), or the pathwise derivative estimator (this estimator is also referred to in the literature as the re-parametrisation trick, infinit”
Assumed Density Filtering Methods for Learning Bayesian Neural Networks (2016, methodology): “We utilize recently proposed stochastic approximation techniques that directly provide unbiased estimates of φln Z through a Monte Carlo approximation (Paisley, Blei, and Jordan 2012; Kingma and Welling 2013; Rezende, Mohamed, and Wierstra 2014; Titsias and Lázaro-Gredilla 2014).”
Hierarchical Variational Models (2015, methodology): “(Titsias and Lázaro-Gredilla, 2014) COPULA VI (Tran et al., 2015) O ( d by making both the parameters of the variational model and recursive posterior approximation functions of their conditioning sets.”
Automatic Differentiation Variational Inference (2016, methodology): “We scale ADVI to large datasets using stochastic optimization with data subsampling (Hoffman et al., 2013; Titsias and Lázaro-Gredilla, 2014).”
The Generalized Reparameterization Gradient (2016, methodology): “This has enabled the application of VI on nonconjugate probabilistic models (Carbonetto et al., 2009; Paisley et al., 2012; Ranganath et al., 2014; Titsias and Lázaro-Gredilla, 2014), deep neural networks (Neal, 1992; Hin-ton et al., 1995; Mnih and Gregor, 2014; Kingma and Welling, 2014), and…”
Topic Modeling in Embedding Spaces (2019, methodology): “We use stochastic optimization, forming noisy gradients by taking Monte Carlo approximations of the full gradient through the reparameterization trick (Kingma and Welling, 2014; Titsias and Lázaro-Gredilla, 2014; Rezende et al., 2014).”
Estimating Heterogeneous Consumer Preferences for Restaurants and Travel Time Using Mobile Location Data (2018, methodology): “In particular, we use the reparameterization gradient (Kingma and Welling, 2014; Titsias and Lázaro-Gredilla, 2014; Rezende, Mohamed and Wierstra, 2014).”
Local Expectation Gradients for Black Box Variational Inference (2015, methodology): “Furthermore, it has been shown experimentally in several studies [8, 15, 20, 9] that the estimate in (6) has relatively low variance and can lead to efficient optimization even when a single sample is used at each iteration.”
Doubly Semi-Implicit Variational Inference (2018, methodology): “A discriminative model with such hierarchical prior may be defined as follows [23, 33, 34, 7, 31, 18]:”