Slice Sampling

By Radford M. Neal. 2000

861 citations recorded by the source corpus.

Markov chain sampling methods that automatically adapt to characteristics of the distribution being sampled can be constructed by exploiting the principle that one can sample from a distribution by sampling uniformly from the region under the plot of its density function. A Markov chain that converges to this uniform distribution can be constructed by alternating uniform sampling in the vertical direction with uniform sampling from the horizontal `slice' defined by the current vertical position, or more generally, with some update that leaves the uniform distribution over this slice invariant. Variations on such `slice sampling' methods are easily implemented for univariate distributions, and can be used to sample from a multivariate distribution by updating each variable in turn. This approach is often easier to implement than Gibbs sampling, and more efficient than simple Metropolis updates, due to the ability of slice sampling to adaptively choose the magnitude of changes made. It is therefore attractive for routine and automated use. Slice sampling methods that update all variables simultaneously are also possible. These methods can adaptively choose the magnitudes of changes made to each variable, based on the local properties of the density function. More ambitiously, such methods could potentially allow the sampling to adapt to dependencies between variables by constructing local quadratic approximations. Another approach is to improve sampling efficiency by suppressing random walks. This can be done using `overrelaxed' versions of univariate slice sampling procedures, or by using `reflective' multivariate slice sampling methods, which bounce off the edges of the slice.

Subfields: Biophysics, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Probabilistic & Bayesian Methods

Citations in the corpus

Scalable Bayesian Optimization Using Deep Neural Networks (2015, methodology): “We integrate out α and β using slice sampling (Neal, 2000) according to the methodology of Snoek et al. (2012) over the marginal likelihood, which is given by log It is clear that the computational bottleneck of this procedure is the inversion of K .”
The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo (2011, methodology): “NUTS overcomes this issue by means of a recursive algorithm reminiscent of the doubling procedure devised by Neal (2003) for slice sampling.”
Stan: A Probabilistic Programming Language (2017, methodology): “For NUTS, the acceptance statistic is defined as the average acceptance probabilities of the trajectory states in the proposed tree; NUTS itself uses a slice sampling algorithm for rejection (Neal 2003; Hoffman and Gelman 2014).”
Benchmarking the Neural Linear Model for Regression (2019, methodology): “To reduce overﬁtting, the noise variance and prior variance (for the Bayesian linear regression) are subsequently marginalized out by slice sampling (Neal et al., 2003) according to the tractable marginal likelihood, using uniform priors.”
Inference of Cause and Effect with Unsupervised Inverse Regression (2015, methodology): “In our implementation, we approximate the posterior p ( x , θ | y ∗ ) using a Markov Chain Monte Carlo (MCMC) method, slice sampling [Neal, 2003].”
Active Learning of Linear Embeddings for Gaussian Processes (2013, methodology): “…evaluate performance using the symmetrized Kullback– Leibler divergence ( SKLD ) between approximate posteriors and the “true” posterior (obtained using a run of slice sampling (Neal, 2003) with 10 5 samples and 10 4 burn-in); the better the approximate marginalization, the smaller this divergence.”
Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows (2018, methodology): “By change of variables, the conditional density is given by In order to sample from the approximate posterior ˆ p ( θ | x o ) ∝ q φ ( x o | θ ) p ( θ ), we use Markov Chain Monte Carlo (MCMC) in the form of Slice Sampling with axis-aligned updates [55].”
Log-Concavity and Strong Log-Concavity: a review (2014, methodology): “The algorithms of Neal (2003) and Roberts and Rosenthal (2002) involve “slice sampling”; and the algorithms of Lovász and Vempala (2007), Lovasz and Vempala (2006), Lovász and Vempala (2007) are based on random walk methods.”
A Customer Lifetime Value Framework for Customer Selection and Resource Allocation Strategy (2004, methodology): “We used “slice-sampling” (Neal 2000) to obtain random samples.”
Learning Transferable Knowledge Through Embedding Spaces (2019, methodology): “The idea behind the SWD is based on the slice sampling [132].”
Inference for Mixtures of Finite Polya Tree Models (2006, methodology): “Slice sampling (Neal 2003) provides an alternative to using the Metropolis–Hastings algorithm and can be used to almost entirely automate the process, although slice sampling will be inefficient if there is high correlation among parameters in the posterior.”
Bayesian Nonparametrics: Hierarchical Bayesian nonparametric models with applications (2010, background): “(2008) proposed a slice sampling approach which adaptively limits the number of states to a finite number (Neal, 2003; Walker, 2007).”