Deep convolutional Gaussian processes

By Kenneth Blomqvist, Samuel Kaski, M. Heinonen. 2018

65 citations recorded by the source corpus.

We propose deep convolutional Gaussian processes, a deep Gaussian process architecture with convolutional structure. Œe model is a principled Bayesian framework for detecting hierarchical combinations of local features for image classi€cation. We demonstrate greatly improved image classi€cation performance compared to current Gaussian process approaches on the MNIST and CIFAR-10 datasets. In particular, we improve CIFAR-10 accuracy by over 10 percentage points.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Greg Yang
2. Ruslan Salakhutdinov
3. Lechao Xiao
4. Jeffrey Pennington
5. Simon Shaolei Du
6. Jaehoon Lee
7. Yuanzhi Li
8. Jascha Narain Sohl-Dickstein

In Computer Vision and Pattern Recognition

1. Greg Yang
2. Lechao Xiao
3. Ruslan Salakhutdinov
4. Jeffrey Pennington
5. Sanjeev Arora
6. Jaehoon Lee
7. Simon Shaolei Du
8. Jascha Narain Sohl-Dickstein

In Representation & Self-Supervised Learning

1. Greg Yang
2. Ruslan Salakhutdinov
3. Sanjeev Arora
4. Jeffrey Pennington
5. Leland McInnes
6. John Healy
7. Jascha Narain Sohl-Dickstein
8. Lechao Xiao

In Probabilistic & Bayesian Methods

1. Greg Yang
2. Jascha Narain Sohl-Dickstein
3. Jaehoon Lee
4. Lechao Xiao
5. Roman Novak
6. Jeffrey Pennington
7. Ruslan Salakhutdinov
8. Yoshua Bengio

Citations in the corpus

• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction (2018, methodology): “Based upon preliminary releases of a so‰ware implementation, UMAP has already found widespread use in the €elds of bioinformatics [4, 15, 37, 2, 36, 13], materials science [27, 19], and machine learning [8, 20, 17, 38] among others.”
• On Exact Computation with an Infinitely Wide Neural Net (2019, background): “Another line of work built probabilistic graphical models (PGMs) which uses kernel as a component [Damianou and Lawrence, 2013, Lawrence and Moore, 2007, Van der Wilk et al., 2017, Kumar et al., 2018, Blomqvist et al., 2018].”
• Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation (2019, methodology): “, 2018; Blomqvist et al., 2018; Borovykh, 2018). In particular, Lee et al. (2018); Garriga-Alonso et al. (2018); Novak et al. (2018) directly converted DNN to GP using this correspondence. Lee et al. (2018) constructed the state-of-the-art (SOTA) permutation-invariant GP on MNIST, and Novak et al. (”
• Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes (2018, background): “(2017) incorporates convolutional structure into GP kernels, with follow-up work stacking multiple such GPs (Kumar et al., 2018; Blomqvist et al., 2018) to produce a deep convolutional GP (which is no longer a GP).”
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, unknown): “[160], Tran et al.”
• A Fine-Grained Spectral Perspective on Neural Networks (2019, background): “This connection was exploited implicitly or explicitly to build new models [15, 32, 16, 61, 62, 12, 58, 31, 10, 11, 20, 39, 34].”
• Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes (2019, background): “Others have exploited this NN-GP correspondence implicitly or explicitly to build new models [11, 33, 12, 57, 58, 7, 54, 32, 4, 6, 18, 43].”