Memory-Based Neighbourhood Embedding for Visual Recognition

By Suichan Li, Dapeng Chen, Bin Liu, Nenghai Yu, Rui Zhao. 2019

42 citations recorded by the source corpus.

Learning discriminative image feature embeddings is of great importance to visual recognition. To achieve better feature embeddings, most current methods focus on designing different network structures or loss functions, and the estimated feature embeddings are usually only related to the input images. In this paper, we propose Memory-based Neighbourhood Embedding (MNE) to enhance a general CNN feature by considering its neighbourhood. The method aims to solve two critical problems, i.e., how to acquire more relevant neighbours in the network training and how to aggregate the neighbourhood information for a more discriminative embedding. We first augment an episodic memory module into the network, which can provide more relevant neighbours for both training and testing. Then the neighbours are organized in a tree graph with the target instance as the root node. The neighbourhood information is gradually aggregated to the root node in a bottom-up manner, and aggregation weights are supervised by the class relationships between the nodes. We apply MNE on image search and few shot learning tasks. Extensive ablation studies demonstrate the effectiveness of each component, and our method significantly outperforms the state-of-the-art approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinguo Zhu
2. Shijie Yu
3. Dapeng Chen
4. Shixiang Tang
5. Wanli Ouyang
6. Xiaogang Wang
7. Dacheng Tao
8. Liu Liu

In Computer Vision and Pattern Recognition

1. Dapeng Chen
2. Shixiang Tang
3. Jinguo Zhu
4. Shijie Yu
5. Wanli Ouyang
6. Mingzhe Rong
7. Yakun Liu
8. Aijun Yang

Citations in the corpus

• Layerwise Optimization by Gradient Decomposition for Continual Learning (2021, positive): “Layerwise gradient update strategy is also applied in meta-learning for rapid loss convergence on transfer learning [13, 14, 48] and few-shot learning [49, 28].”
• Complementary Relation Contrastive Distillation (2021, background): “There are a few recent works analyzing and exploiting the mutual relation between data samples [28, 34, 32, 33, 6, 5, 7, 25].”