One-Shot Generalization in Deep Generative Models

By Danilo Jimenez Rezende, Shakir Mohamed, Ivo Danihelka, Karol Gregor, Daan Wierstra. 2016

261 citations recorded by the source corpus.

Humans have an impressive ability to reason about new concepts and experiences from just a single example. In particular, humans have an ability for one-shot generalization: an ability to encounter a new concept, understand its structure, and then be able to generate compelling alternative variations of the concept. We develop machine learning systems with this important capacity by developing new deep generative models, models that combine the representational power of deep learning with the inferential power of Bayesian reasoning. We develop a class of sequential generative models that are built on the principles of feedback and attention. These two characteristics lead to generative models that are among the state-of-the art in density estimation and image generation. We demonstrate the one-shot generalization ability of our models using three tasks: unconditional sampling, generating new exemplars of a given concept, and generating new exemplars of a family of concepts. In all cases our models are able to generate compelling and diverse samples-- having seen new examples just once--providing an important class of general-purpose models for one-shot machine learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Can Active Memory Replace Attention? (2016, background): “Image models can benefit from attention too; for instance, image captioning models can focus on the relevant parts of the image when describing it [8]; generative models for images yield especially good results with 30th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona,…”
Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions (2017, background): “Rezende et al. (2016) develop a sequential generative model for few-shot learning, generalizing the Deep Recurrent Attention Writer (DRAW) model (Gregor et al., 2015).”
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (2017, methodology): “Few-shot learning methods have also been developed for speciﬁc tasks such as generative modeling (Edwards & Storkey, 2017; Rezende et al., 2016) and image recognition (Vinyals et al., 2016).”
One-Shot Imitation Learning (2017, background): “…been studied for image recognition (Vinyals et al., 2016; Koch, 2015; Santoro et al., 2016; Ravi & Larochelle, 2017), generative modeling (Edwards & Storkey, 2017; Rezende et al., 2016), learning “fast” reinforcement learning agents with recurrent policies (Duan et al., 2016; Wang et al., 2016).”
Learning deep representations by mutual information estimation and maximization (2018, methodology): “Generative models in representation learning Generative models are commonly used as methods for building representations (Vincent et al., 2010; Kingma et al., 2014; Salimans et al., 2016; Rezende et al., 2016; Donahue et al., 2016), and mutual information plays an important role in their representat”
Learning feed-forward one-shot learners (2016, background): “These problems are usually tackled by using generative models [18, 13] or, in a discriminative setting, using ad-hoc solutions such as exemplar support vector machines (SVMs) [14].”
Data Augmentation Generative Adversarial Networks (2017, methodology): “Others have used nonparametric Bayesian approaches (Salakhutdinov et al., 2012), and conditional variational autoencoders (Rezende et al., 2016).”
Meta-SGD: Learning to Learn Quickly for Few Shot Learning (2017, background): “This motivates a lot of studies under the name of few-shot learning that aims to learn from only a few examples, which asks for new approaches to machine learning [16, 19, 20].”
Towards a Neural Statistician (2016, methodology): “For example since the instance encoder can approximate a polynomial on a compact domain, and so can the post-pooling network, a statistic network can approximate any moment of a distribution.”
Conditional Neural Processes (2018, background): “Another successful latent variable approach is to explicitly condition on some context during inference (J. Rezende et al., 2016).”
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation (2016, background): “There has been recent work [8, 11, 25, 18, 41, 12, 16] in using deep generative models to disentangle multiple factors of variations (objects, pose, location, etc) from pixel data.”
Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches (2022, methodology): “Keeping the feature extractor F θ ﬁxed, we train the few-shot weight generator G ψ taking inspiration from few-shot learning literature [50, 40, 48].”
Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning (2018, background): “To reduce the amount of data needed for training, a recent surge of research interest has focused on fewshot learning (Lake, Salakhutdinov, and Tenenbaum 2015; Rezende et al. 2016) – to learn a classification model with very few examples from each class.”
Sample Efficient Adaptive Text-to-Speech (2018, methodology): “Rezende et al. (2016) developed a sequential generative model that extended the Deep Recurrent Attention Writer (DRAW) model (Gregor et al., 2015), and Reed et al. (2018) extended PixelCNN (Van Oord et al., 2016) with neural attention for few-shot auto-regressive density modeling.”