Deep Prior

By Alexandre Lacoste, Thomas Boquet, Negar Rostamzadeh, Boris N. Oreshkin, Wonchang Chung, David Krueger. 2017

7 citations recorded by the source corpus.

The recent literature on deep learning offers new tools to learn a rich probability distribution over high dimensional data such as images or sounds. In this work we investigate the possibility of learning the prior distribution over neural network parameters using such tools. Our resulting variational Bayes algorithm generalizes well to new tasks, even when very few training examples are provided. Furthermore, this learned prior allows the model to extrapolate correctly far from a given task's training data on a meta-dataset of periodic signals.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Probabilistic Model-Agnostic Meta-Learning (2018, methodology): “Our work is most related to methods that combine deep networks and probabilistic methods for few-shot learning [6, 15, 23].”
TADAM: Task dependent adaptive metric for improved few-shot learning (2018, neutral): “[12] who rely on hierarchical Bayesian modeling.”
Evaluation Gaps in Machine Learning Practice (2022, background): “Consider Bayesian approaches to dealing with uncertainty about data distributions [90, 98, 115], especially when sample sizes are small or prior work has revealed systematic biases.”
Adaptive Cross-Modal Few-Shot Learning (2019, background): “Given the universality of MAML, many follow-up works were recently proposed to improve its performance on few-shot learning [33, 21].”