Variational Implicit Processes

By Chao Ma, Yingzhen Li, José Miguel Hernández-Lobato. 2018

77 citations recorded by the source corpus.

We introduce the implicit processes (IPs), a stochastic process that places implicitly defined multivariate distributions over any finite collections of random variables. IPs are therefore highly flexible implicit priors over functions, with examples including data simulators, Bayesian neural networks and non-linear transformations of stochastic processes. A novel and efficient approximate inference algorithm for IPs, namely the variational implicit processes (VIPs), is derived using generalised wake-sleep updates. This method returns simple update equations and allows scalable hyper-parameter learning with stochastic optimization. Experiments show that VIPs return better uncertainty estimates and lower errors over existing inference methods for challenging models such as Bayesian neural networks, and Gaussian processes.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

Neural Processes (2018, background): “Finally concurrent work by Ma et al introduces variational implicit processes, which share large part of the motivation of NPs but are implemented as GPs (Ma et al., 2018).”
Physics-informed deep generative models (2018, unknown): “Hence, the ability to quantify predictive uncertainty and learn in a sample-efﬁcient manner is a necessity [5–8], especially in data-limited domains [9].”
Functional Variational Bayesian Neural Networks (2019, background): “Variational Implicit Processes (VIP) (Ma et al., 2018) are, in a sense, the reverse of fBNNs: they specify BNN priors and use GPs to approximate the posterior.”
Understanding Variational Inference in Function-Space (2020, methodology): “…the predictions matter, a recent line of work has focused on directly approximating the posterior predictive distribution in ‘function-space’ (Sun et al., 2018; Ma et al., 2019; Wang et al., 2019), with the intent of reducing the inﬂuence of the speciﬁc parameterization on the quality of inference.”
Attentive Neural Processes (2019, methodology): “Variational Implicit Processes (VIP) (Ma et al., 2018) are also related to NPs, where VIP deﬁnes a stochastic process using the same decoder setup with a ﬁnite dimensional z .”