Grafit: Learning fine-grained image representations with coarse labels

By Hugo Touvron, Alexandre Sablayrolles, Matthijs Douze, Matthieu Cord, Herv'e J'egou. 2020

83 citations recorded by the source corpus.

This paper tackles the problem of learning a finer representation than the one provided by training labels. This enables fine-grained category retrieval of images in a collection annotated with coarse labels only.Our network is learned with a nearest-neighbor classifier objective, and an instance loss inspired by self-supervised learning. By jointly leveraging the coarse labels and the underlying fine-grained latent space, it significantly improves the accuracy of category-level retrieval methods.Our strategy outperforms all competing methods for retrieving or classifying images at a finer granularity than that available at train time. It also improves the accuracy for transfer learning tasks to fine-grained datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dacheng Tao
2. Jing Zhang
3. Yunhe Wang
4. Kai Han
5. Hugo Touvron
6. Chunjing Xu
7. Herv'e J'egou
8. Jianyuan Guo

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Yunhe Wang
3. Kai Han
4. Jing Zhang
5. Luc Van Gool
6. Jianyuan Guo
7. Chunjing Xu
8. Jiashi Feng

Citations in the corpus

• Transformer in Transformer (2021, methodology): “Model Params (M) ImageNet CIFAR10 CIFAR100 Flowers Pets iNat-19 CNN-based Grafit ResNet-50 [32] 25.”
• Instance-Conditioned GAN (2021, unknown): “We then define the set A i of k nearest neighbors for each data sample using the cosine similarity – as is common in nearest neighbor classifiers, e.g. [53, 54] – over the features h i .”
• DeiT III: Revenge of the ViT (2022, unknown): “Model CIFAR-10 CIFAR-100 Flowers Cars iNat-18 iNat-19 Grafit ResNet-50 [52] 98.”
• AutoFormer: Searching Transformers for Visual Recognition (2021, unknown): “5 - CNN Manual Grafit RegNetY-8GF [51] 39M 23.”
• ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond (2022, background): “Model Params (M) Cifar10 Cifar100 iNat19 Cars Flowers Pets Grafit ResNet-50 [73] 25.”
• ResNet strikes back: An improved training procedure in timm (2021, background): “[48] Hugo Touvron, Alexandre Sablayrolles, M.”