Improving Generalization via Scalable Neighborhood Component Analysis

By Zhirong Wu, Alexei A. Efros, Stella X. Yu. 2018

153 citations recorded by the source corpus.

Current major approaches to visual recognition follow an end-to-end formulation that classifies an input image into one of the pre-determined set of semantic categories. Parametric softmax classifiers are a common choice for such a closed world with fixed categories, especially when big labeled data is available during training. However, this becomes problematic for open-set scenarios where new categories are encountered with very few examples for learning a generalizable parametric classifier. We adopt a non-parametric approach for visual recognition by optimizing feature embeddings instead of parametric classifiers. We use a deep neural network to learn the visual feature that preserves the neighborhood structure in the semantic space, based on the Neighborhood Component Analysis (NCA) criterion. Limited by its computational bottlenecks, we devise a mechanism to use augmented memory to scale NCA for large datasets and very deep networks. Our experiments deliver not only remarkable performance on ImageNet classification for such a simple non-parametric method, but most importantly a more generalizable feature representation for sub-category discovery and few-shot recognition.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Media Technology, Representation & Self-Supervised Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Media Technology

In Representation & Self-Supervised Learning

In Optimization & Training

Citations in the corpus

Grafit: Learning fine-grained image representations with coarse labels (2020, methodology): “The rows corresponding to the main baselines and meth-ods discussed through our paper are highlighted: our baseline and improved SNCA+ in grey and red, and our two variants Graﬁt-FC and Graﬁt in blue.”
Local Aggregation for Unsupervised Learning of Visual Embeddings (2019, methodology): “As shown empirically in [71] (albeit in the supervised context), as long as the scaling parameter τ 1, maximizing P (A|vi) for any set A causes the emergence of natural “sub-categories” in (the embeddings of) A, and encourages vi to move closer to one of these sub-categories rather than their overal”
What makes for good views for contrastive learning (2020, background): “Leveraging labeled data in contrastive representation learning has been shown to guide representations towards task-relevant features that improve performance [61, 25, 29, 57].”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, neutral): “Our instance discrimination loss is inspired by previous works [96, 94, 40, 17, 32, 46].”
Constrained Few-shot Class-incremental Learning (2022, background): “Further, the size of the EM is set by c vectors [42, 50], or c× k vectors [21, 53, 54] where c and k represent the number of classes and training examples per class, respectively.”
Supervised Contrastive Learning (2020, positive): “We further improve on [53] by the increased use of data augmentation, a disposable contrastive head and two-stage training (contrastive followed by crossentropy), and crucially, changing the form of the loss function to significantly improve results (see Section 3).”
Unsupervised Learning From Video With Deep Neural Embeddings (2019, background): “However, recent advances in learning with deep visual embeddings have begun to produce unsupervised representations that rival the visual task transfer power of representations learned by their supervised counterparts [37, 36, 40, 3].”
CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning (2021, methodology): “We also evaluate CReST on ImageNet127 [17, 45] to verify its performance on large-scale datasets.”
Negative Margin Matters: Understanding Margin in Few-shot Classification (2020, unknown): “ResNet-18 SNCA [51] 57.”
Training Vision Transformers for Image Retrieval (2021, unknown): “The DeiT-S 128 variant with compressed representation outperforms all previous methods except for the ProxyNCA++ model that uses 2048-”
Finding Task-Relevant Features for Few-Shot Learning by Category Traversal (2019, unknown): “[39] gives a 61.”
Generalized Key-Value Memory to Flexibly Adjust Redundancy in Memory-Augmented Networks (2022, background): “These MANNs are typically applied in knowledge-based reasoning [2]–[4], sequential prediction [5], unsupervised learning [6], [7], and few-shot learning tasks [8], [9].”
Supervision Accelerates Pre-training in Contrastive Semi-Supervised Learning of Visual Representations. (2020, background): “Most similar to the SuNCEt loss is that of Wu et al. [51], which investigates neighborhood component analysis (NCA) [18] in the fully supervised setting.”
Deep Metric Transfer for Label Propagation with Limited Annotated Data (2018, methodology): “After label propagation and finetuning on the new data, our supervised propagation obtains a significant boost of 6% over SNCA.”
Rethinking Semantic Segmentation: A Prototype View (2022, background): “This hurts generalizability [115], especially in the large-vocabulary case; for instance, if there are 800 classes andD=512, we need 0.”
Deep Learning on Small Datasets without Pre-Training using Cosine Loss (2019, background): “Metric learning techniques are common in this scenario, which aim at learning discriminative features from a large dataset that generalize well to new classes [45, 38, 41, 47, 50], so that classification in face of limited data can be performed with a nearest neighbor search.”
Asymmetric metric learning for knowledge transfer (2020, background): “The large number of potential tuples gives rise to mining [20, 75] and memory [73, 76] mechanisms.”