Approximate Gaussian Integration using Expectation Propagation

By John P. Cunningham, Philipp Hennig, Simon Lacoste-Julien. 2011

40 citations recorded by the source corpus.

While Gaussian probability densities are omnipresent in applied mathematics, Gaussian cumulative probabilities are hard to calculate in any but the univariate case. We offer here an empirical study of the utility of Expectation Propagation (EP) as an approximate integration method for this problem. For rectangular integration regions, the approximation is highly accurate. We also extend the derivations to the more general case of polyhedral integration regions. However, we find that in this polyhedral case, EP's answer, though often accurate, can be almost arbitrarily wrong. These unexpected results elucidate an interesting and non-obvious feature of EP not yet studied in detail, both for the problem of Gaussian probabilities and for EP more generally.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets (2016, methodology): “This particular application of EP to Gaussian integrals was introduced by Cunningham et al. [39] where all the details can be found.”
Entropy Search for Information-Efficient Global Optimization (2011, methodology): “We refer interested readers to recent work by Cunningham et al. (2011), which gives a detailed description of these aspects.”
Black-box $\alpha$-divergence Minimization (2015, methodology): “For instance, VB provides poor approximations when non-smooth likelihood functions are used (Cunningham et al., 2011; Turner & Sahani, 2011b; Opper & Winther, 2005).”
Stochastic Expectation Propagation (2015, background): “At first sight, it therefore appears well suited to large-data problems: the locality of computation make the algorithm simple to parallelise and distribute, and good practical performance on a range of small data applications suggest that it will be accurate [9, 10, 11].”
Bayesian Optimization with Inequality Constraints (2014, methodology): “In the case of dependent constraints, this multivariate Gaussian probability can be calculated with available numerical methods (Cunningham et al., 2011).”
Fast Bayesian hyperparameter optimization on large datasets (2017, methodology): “This particular application of EP (dubbed EPMGP) to Gaussian integrals was introduced by Cunningham et al. (2012) where all the details can be found.”
Expectation Propagation as a Way of Life (2020, methodology): “…problems where computational cost is an issue, notably for Gaussian processes (Rasmussen, and Williams, 2006, Jylänki, Vanhatalo, and Vehtari, 2011, Cunningham, Hennig, and Lacoste-Julien, 2011, and Vanhatalo et al., 2013), and efforts are made to keep the algorithm stable as well as fast.”