CNN-RNN: a large-scale hierarchical image classification framework
By Yanming Guo, Y. Liu, Erwin M. Bakker, Yuanhao Guo, Michael S. Lew. 2017
144 citations recorded by the source corpus.
Objects are often organized in a semantic hierarchy of categories, where fine-level categories are grouped into coarse-level categories according to their semantic relations. While previous works usually only classify objects into the leaf categories, we argue that generating hierarchical labels can actually describe how the leaf categories evolved from higher level coarse-grained categories, thus can provide a better understanding of the objects. In this paper, we propose to utilize the CNN-RNN framework to address the hierarchical image classification task. CNN allows us to obtain discriminative features for the input images, and RNN enables us to jointly optimize the classification of coarse and fine labels. This framework can not only generate hierarchical labels for images, but also improve the traditional leaf-level classification performance due to incorporating the hierarchical information. Moreover, this framework can be built on top of any CNN architecture which is primarily designed for leaf-level classification. Accordingly, we build a high performance network based on the CNN-RNN paradigm which outperforms the original CNN (wider-ResNet) and also the current state-of-the-art. In addition, we investigate how to utilize the CNN-RNN framework to improve the fine category classification when a fraction of the training data is only annotated with coarse labels. Experimental results demonstrate that CNN-RNN can use the coarse-labeled training data to improve the classification of fine categories, and in some cases it even surpasses the performance achieved by fully annotated training data. This reveals that, CNN-RNN can alleviate the challenge of specialized and expensive annotation of fine labels.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Hugo Touvron
- Matthieu Cord
- Peter Hase
- Herv'e J'egou
- Alexandre Sablayrolles
- Cynthia Rudin
- Matthijs Douze
- Oscar Li
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Matthieu Cord
- Herv'e J'egou
- Dacheng Tao
- Jiashi Feng
- Trevor Darrell
- Andrew Zisserman
- Cordelia Schmid
In Model Architectures
- Herv'e J'egou
- Hugo Touvron
- Matthieu Cord
- Peter Hase
- Hadi Kazemi
- Matthijs Douze
- Oscar Li
- Alexandre Sablayrolles
Citations in the corpus
- Grafit: Learning fine-grained image representations with coarse labels (2020, background): “In some works on hierarchical image classification [19, 26, 45, 49], a coarse annotation is available for all training images, but only a subset of the training images are labelled at a fine granularity.”
- A Weakly Supervised Fine Label Classifier Enhanced by Coarse Supervision (2019, methodology): “We arrange our experimental setup as described in [28] and [13].”
- Interpretable Image Recognition with Hierarchical Prototypes (2019, methodology): “…and Torralba 2012), CNNs, (Ahmed and Torresani 2017; Redmon and Farhadi 2017; Zhu and Bain 2017; Zhao, Li, and Xing 2011; Kuang et al. 2018; Yan et al. 2015; Tamaazousti, Borgne, and Hudelot 2017; Roy, Panda, and Roy 2018), and recently the use of a CNN and RNN together (Guo et al. 2017).”