Convolutional Gaussian Processes

By Mark van der Wilk, Carl E. Rasmussen, James Hensman. 2017

137 citations recorded by the source corpus.

We present a practical way of introducing convolutional structure into Gaussian processes, making them more suited to high-dimensional inputs like images. The main contribution of our work is the construction of an inter-domain inducing point approximation that is well-tailored to the convolutional kernel. This allows us to gain the generalisation benefit of a convolutional kernel, together with fast but accurate posterior inference. We investigate several variations of the convolutional kernel, and apply it to MNIST and CIFAR-10, which have both been known to be challenging for Gaussian processes. We also show how the marginal likelihood can be used to find an optimal weighting between convolutional and RBF kernels to further improve performance. We hope that this illustration of the usefulness of a marginal likelihood will help automate discovering architectures in larger models.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data (2019, background): “The recent advances to scale GPs to highdimensional input include Kronecker product decomposition that exploits the spatial structure [1, 4, 5], convolutional kernels [6] and other algorithmic and software developments [7].”
Calibrating Deep Convolutional Gaussian Processes (2018, methodology): “Center: CGP [27].”
On Exact Computation with an Infinitely Wide Neural Net (2019, background): “Another line of work built probabilistic graphical models (PGMs) which uses kernel as a component [Damianou and Lawrence, 2013, Lawrence and Moore, 2007, Van der Wilk et al., 2017, Kumar et al., 2018, Blomqvist et al., 2018].”
Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation (2019, background): “There is a long line of work investigating random classic spiking or hopfield networks, for example Landau & Sompolinsky (2018); Crisanti & Sompolinsky (2018); Kadmon & Sompolinsky (2015); Stern et al.”
Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes (2018, background): “[39] incorporates convolutional structure into GP kernels, with follow-up work stacking multiple such GPs [20, 4, 2] to produce a deep convolutional GP (which is no longer a GP).”
When Gaussian Process Meets Big Data: A Review of Scalable GPs (2018, methodology): “The recently developed convolutional kernel [171] opens up the way of DGP for CV tasks [202].”
FedLoc: Federated Learning Framework for Data-Driven Cooperative Localization and Location Data Processing (2020, background): “The recently proposed graph GP provides another way for learning from high-dimensional data points living on nonEuclidean domains, see for instance [115, 116, 117].”
Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes (2019, background): “[54] Mark van der Wilk, Carl Edward Rasmussen, and James Hensman.”
A Fine-Grained Spectral Perspective on Neural Networks (2019, background): “This connection was exploited implicitly or explicitly to build new models [14, 31, 15, 60, 61, 11, 57, 30, 9, 10, 19, 38, 33].”
Deep convolutional Gaussian processes (2018, methodology): “Recently Van der Wilk et al. (2017) proposed the rst convolution-based Gaussian process for images with promising performance. ey proposed a weighted additive model where Gaussian process responses over image subpatches are aggregated for image classi cation. e convolutional Gaussian process is…”
Deep Convolutional Networks as shallow Gaussian Processes (2018, methodology): “Second, it is possible to deﬁne a convolutional GP [14] or a deep convolutional GP [15] by deﬁning a GP that takes an image patch as input, and using that GP as a component in a larger CNN-like system.”
Variational Implicit Processes (2018, background): “…18], where compositions of GP priors are proposed to represent prior over compositional functions; (2) the search and design of kernels for accurate and efﬁcient learning [3, 5, 24, 79, 86, 89], and (3) deep kernel learning that uses deep neural net features as the inputs to GPs [1,10,40,43,92].”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, unknown): “[159], Blomqvist et al.”