Meta-Neighborhoods
By Siyuan Shan, Yang Li, Junier B. Oliva. 2019
17 citations recorded by the source corpus.
Traditional methods for training neural networks use training data just once, as it is discarded after training. Instead, in this work we also leverage the training data during testing to adjust the network and gain more expressivity. Our approach, named Meta-Neighborhoods, is developed under a multi-task learning framework and is a generalization of k-nearest neighbors methods. It can flexibly adapt network parameters w.r.t. different query data using their respective local neighborhood information. Local information is learned and stored in a dictionary of learnable neighbors rather than directly retrieved from the training set for greater flexibility and performance. The network parameters and the dictionary are optimized end-to-end via meta-learning. Extensive experiments demonstrate that Meta-Neighborhoods consistently improved classification and regression performance across various network architectures and datasets. We also observed superior improvements than other state-of-the-art meta-learning methods designed to improve supervised learning.
Subfields: Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Meta & Continual Learning
- Chelsea Finn
- Sergey Levine
- Abhishek Gupta
- Han-Jia Ye
- Wei-Lun Chao
- Pieter Abbeel
- Jorge Armando Mendez Mendez
- Eric Eaton
Citations in the corpus
- Dynamic Neural Networks: A Survey (2021, background): “Instead of adjusting weights conditioned on every sample itself, meta-neighborhoods [113] adapt the network parameters to each input sample based on its similarity to the neighbors stored in a dictionary.”
- Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift (2020, methodology): “Some meta-learning methods adapt using both labeled and unlabeled data, such as [53, 71, 46, 2, 39, 4], or consider the setting where task groupings are not known, such as [23, 30, 24, 59], though these works do not focus on the same setting of distribution shift.”