Few-Shot Learning with Graph Neural Networks
By Victor Garcia Satorras, Joan Bruna. 2017
1,368 citations recorded by the source corpus.
We propose to study the problem of few-shot learning with the prism of inference on a partially observed graphical model, constructed from a collection of input images whose label can be either observed or not. By assimilating generic message-passing inference algorithms with their neural-network counterparts, we define a graph neural network architecture that generalizes several of the recently proposed few-shot learning models. Besides providing improved numerical performance, our framework is easily extended to variants of few-shot learning, such as semi-supervised or active learning, demonstrating the ability of graph-based models to operate well on 'relational' tasks.
Subfields: Artificial Intelligence, Graph Learning, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Sergey Levine
- Mohit Bansal
- Dacheng Tao
- Trevor Darrell
- Quoc V. Le
- Jianfeng Gao
- Percy Liang
In Graph Learning
- William L. Hamilton
- Jure Leskovec
- Max Welling
- Peter W. Battaglia
- Rex Ying
- Stephan Günnemann
- Raquel Urtasun
- Marc Brockschmidt
In Meta & Continual Learning
Citations in the corpus
- CrossTransformers: spatially-aware few-shot transfer (2020, background): “Matching-based methods [28, 75, 78, 89] instead learn representations for similarity functions [11, 16, 17, 44, 82], in the hope that the similarities will generalize to new data.”
- Meta-Learning Probabilistic Inference for Prediction (2018, background): “…pre-training and/or residual networks (Bauer et al., 2017; Qiao et al., 2017; Rusu et al., 2018; Gidaris and Komodakis, 2018; Oreshkin et al., 2018; Garcia and Bruna, 2017; Lacoste et al., 2018) have been excluded so that the quality of the learning algorithm can be assessed separately from the…”
- Meta-learning with differentiable closed-form solvers (2018, methodology): “4 % 99.9 ± 0.1% 95.8 ± 0.3% 98.9 ± 0.2% P ROTO NET (Snell et al., 2017) 98.5 ± 0.2% 99.5 ± 0.1% 95.3 ± 0.2% 98.7 ± 0.1% SNAIL (Mishra et al., 2018) 99.07 ± 0.16% 99.77 ± 0.09% 97.64 ± 0.30% 99.36 ± 0.18% GNN (Garcia & Bruna, 2018) 99 (Snell et al., 2017) 71.7 ± 1.0% 84.8 ± 0.7% 76.4 ± 0.9% 88.5 ± 0.”
- Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples (2019, methodology): “These were based on graph neural networks (Satorras & Estrach, 2018), relational networks (Sung et al., 2018) and attentional networks (Mishra et al., 2018).”
- A Closer Look at Few-shot Classification (2019, background): “The problem of learning to generalize to unseen classes during training, known as few-shot clas-sification , has attracted considerable attention Vinyals et al. (2016); Snell et al. (2017); Finn et al. (2017); Ravi & Larochelle (2017); Sung et al. (2018); Garcia & Bruna (2018); Qi et al. (2018).”
- Relational inductive biases, deep learning, and graph networks (2018, background): “They have been effective at tasks thought to have rich relational structure, such as visual scene understanding tasks (Raposo et al., 2017; Santoro et al., 2017) and few-shot learning (Garcia and Bruna, 2018).”
- Graphite: Iterative Generative Modeling of Graphs (2018, background): “The tasks which have benefited from graph neural networks, including semi-supervised learning [23] and few shot learning [9], involve encoding an input graph to a final output representation (such as the labels associated with the nodes).”
- Probabilistic Model-Agnostic Meta-Learning (2018, neutral): “78% GNN [12] 50 .”
- Few-Shot and Continual Learning with Attentive Independent Mechanisms (2021, methodology): “A generalization of all the mentioned work is done by modeling the metric using a graph neural network proposed by [13].”
- Incremental Few-Shot Learning with Attention Attractor Networks (2018, background): “Recently, Garcia & Bruna (2018) propose a graph neural networks based method which captures the information propagation from the labeled support set to the query set.”
- FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (2018, methodology): “By adapting them to relation classification, we also evaluate four recently proposed few-shot learning methods, including Meta Network (Munkhdalai and Yu, 2017), GNN (Satorras and Estrach, 2018), SNAIL (Mishra et al., 2018), and Prototypical Network ( Snell et al., 2017).”
- Meta-Learning in Neural Networks: A Survey (2020, methodology): “Vanilla methods in this family have been further enhanced by making the embedding task-conditional [99], [115], learning a more elaborate comparison metric [89], [90], or combiningwith gradient-basedmeta-learning to train other hyper-parameters such as stochastic regularizers [116].”
- A Comprehensive Survey on Graph Neural Networks (2019, background): “It includes human-object interaction [144], few-shot image classification [145], [146], [147], semantic segmentation [148], [149], visual reasoning [150], and question answering [151].”
- A Baseline for Few-Shot Image Classification (2019, background): “Semi-supervised few-shot learning is typically formulated as having access to extra unlabeled data during meta-training or few-shot training [9, 36].”
- Boosting Few-Shot Visual Learning With Self-Supervision (2019, neutral): “, training images, of a few-shot task [26, 33, 53, 55, 57, 59]; methods learning to map a test example to a class label by accessing memory modules that store training examples for that task [11, 25, 35, 38, 50]; approaches that learn how to generate the weights of a classifier [12, 16, 43, 44, 13] ”
- An Ensemble of Epoch-Wise Empirical Bayes for Few-Shot Learning (2019, background): “The metrics include Euclidean distance [64], cosine distance [8,73], relation module [24,41,71] and graph-based similarity [45, 62].”