Neural Adaptive Sequential Monte Carlo

By Shixiang Shane Gu, Zoubin Ghahramani, Richard E. Turner. 2015

156 citations recorded by the source corpus.

Sequential Monte Carlo (SMC), or particle filtering, is a popular class of methods for sampling from an intractable target distribution using a sequence of simpler intermediate distributions. Like other importance sampling-based methods, performance is critically dependent on the proposal distribution: a bad proposal can lead to arbitrarily inaccurate estimates of the target distribution. This paper presents a new method for automatically adapting the proposal using an approximation of the Kullback-Leibler divergence between the true posterior and the proposal distribution. The method is very flexible, applicable to any parameterized proposal distribution and it supports online and batch variants. We use the new framework to adapt powerful proposal distributions with rich parameterizations based upon neural networks leading to Neural Adaptive Sequential Monte Carlo (NASMC). Experiments indicate that NASMC significantly improves inference in a non-linear state space model outperforming adaptive proposal methods including the Extended Kalman and Unscented Particle Filters. Experiments also indicate that improved inference translates into improved parameter learning when NASMC is used as a subroutine of Particle Marginal Metropolis Hastings. Finally we show that NASMC is able to train a latent variable recurrent neural network (LV-RNN) achieving results that compete with the state-of-the-art for polymorphic music modelling. NASMC can be seen as bridging the gap between adaptive SMC methods and the recent work in scalable, black-box variational inference.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Probabilistic & Bayesian Methods

In Signal Processing

Citations in the corpus

Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control (2016, background): “Other approaches have examined RNNs with richer expressivity, latent-variables for notes, or raw audio synthesis (Boulanger-Lewandowski et al., 2012; Gu et al., 2015; Chung et al., 2015).”
Learning and Querying Fast Generative Models for Reinforcement Learning (2018, background): “…models We build directly on a plethora of recent work exploring the continuum of models ranging from standard recurrent neural networks (RNNs) to fully stochastic models with uncertainty (Chung et al., 2015; Archer et al., 2015; Fraccaro et al., 2016; Krishnan et al., 2015; Gu et al., 2015).”
Masked Autoregressive Flow for Density Estimation (2017, background): “The second step would be to approximate Equation (13) with Monte Carlo, using samples {xn} drawn from πx(x), as follows: DKL(pu(u) ‖πu(u)) = Eπx(x) ( log πx(x)− log πu ( f−1(x) ) − log ∣∣∣∣det(∂f−1∂x )∣∣∣∣) (18) ≈ 1 N ∑ n ( log πx(xn)− log πu ( f−1(xn) ) − log ∣∣∣∣det(∂f−1∂x )∣∣∣∣).”
Sequential Neural Models with Stochastic Layers (2016, methodology): “In this section the SRNN is evaluated on the modeling of speech and polyphonic music data, as they have shown to be difficult to model without a good representation of the uncertainty in the latent states [3, 7, 11, 12, 15].”
Structured Inference Networks for Nonlinear State Space Models (2016, neutral): “Compared to LV-RNN (that optimizes the inclusive KL-divergence), DMM-Aug obtains better results on all datasets except JSB.”
Inference Compilation and Universal Probabilistic Programming (2016, background): “This is related, but diﬀerent to the approaches that adapt proposal distributions for the importance sampling family of algorithms (Gu et al., 2015; Cheng and Druzdzel, 2000).”
Iterative Refinement of Approximate Posterior for Training Directed Belief Networks (2015, background): “etter reﬁnement step than AIS but typically requires resampling to control variance. While parametric versions exist that could be applicable to training directed graphical models with discrete units [9, 19], their applicability as a general reﬁnement procedure is limited as the reﬁnement parameters”
Variational Sequential Monte Carlo (2017, methodology): “Gu et al. [2015] learn proposals by minimizing the Kullback-Leibler ( KL ) from the posterior to proposal using SMC samples; this strategy can suffer from high variance when the initial SMC proposal is poor.”
Dynamical Variational Autoencoders: A Comprehensive Review (2020, methodology): “In other words, the authors of (Gu et al., 2015) show that their sampling-based approach can be used to optimize the observed data marginal likelihood, for estimating the generative model parameters in the variational framework.”
Applications of Probabilistic Programming (Master's thesis, 2015) (2016, methodology): “54 3.0.3.1 Functions to generate training and test episodes {y1:T} 55 3.0.3.2 Comparing sequential Monte Carlo runs without and with data-driven proposals . . . . . . . . . . . . . . . 58 x 3.0.4 Applying the approach to the DDPMO model . . . . . . . . . . 60 3.0.4.1 DDPMO model in Anglican . . . . ”
Fast $\epsilon$-free Inference of Simulation Models with Bayesian Conditional Density Estimation (2016, methodology): “Neural density estimators have been employed in learning proposal distributions for importance sampling [24] and Sequential Monte Carlo [11, 23].”
Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives (2018, methodology): “It was extended to the multi-sample setting by (Bornschein & Bengio, 2014) and the sequential setting in (Gu et al., 2015).”
A Universal Marginalizer for Amortized Inference in Generative Models (2017, methodology): “Amortized inference has been popular for Sequential Monte Carlo and has been used to learn in advance either parameters [Gu et al., 2015] or a discriminative model which provides conditional density estimates [Morris, 2001, Paige and Wood, 2016].”
ZhuSuan: A Library for Bayesian Deep Learning (2017, methodology): “This technique proves to be successful in applications like dynamic systems (Gu et al., 2015), inference acceleration (Du et al., 2017; Paige and Wood, 2016) and learning attention models (Ba et al., 2015).”
MultiVerse: Causal Reasoning using Importance Sampling in Probabilistic Programming (2019, methodology): “The standard inference techniques for amortised approximate inference (Gu et al., 2015; Germain et al., 2015; Perov et al., 2015; Paige and Wood, 2016a; Perov, 2016; Le et al., 2016; Morris, 2001; Paige and Wood, 2016b; Ritchie et al., 2016; Le et al., 2017) and in particular importance sampling inf”
Navigation and planning in latent maps (2018, background): “Several works were proposed recently that overcome the intractabilities of Bayes filters via approximate inference (Gu et al., 2015; Krishnan et al., 2015; Karl et al., 2016; Fraccaro et al., 2016; Maddison et al., 2017).”
Tuning Recurrent Neural Networks with Reinforcement Learning (2016, methodology): “We use an LSTM trained on data (the Note RNN) to supply the initial weights for three networks in RL Tuner: the Q-network and Target Q-network in the DQN algorithm as described in Section 2.1, and a Reward RNN.”
Filtering Variational Objectives (2017, methodology): “Other approaches to learning in neural latent variable models include [34], who use importance sampling to approximate gradients under the posterior, and [35], who use sequential Monte Carlo to approximate gradients under the posterior.”