Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification From the Bottom Up

By Weifeng Ge, Xiangru Lin, Yizhou Yu. 2019

296 citations recorded by the source corpus.

Given a training dataset composed of images and corresponding category labels, deep convolutional neural networks show a strong ability in mining discriminative parts for image classification. However, deep convolutional neural networks trained with image level labels only tend to focus on the most discriminative parts while missing other object parts, which could provide complementary information. In this paper, we approach this problem from a different perspective. We build complementary parts models in a weakly supervised manner to retrieve information suppressed by dominant object parts detected by convolutional neural networks. Given image level labels only, we first extract rough object instances by performing weakly supervised object detection and instance segmentation using Mask R-CNN and CRF-based segmentation. Then we estimate and search for the best parts model for each object instance under the principle of preserving as much diversity as possible. In the last stage, we build a bi-directional long short-term memory (LSTM) network to fuze and encode the partial information of these complementary parts into a comprehensive feature for image classification. Experimental results indicate that the proposed method not only achieves significant improvement over our baseline models, but also outperforms state-of-the-art algorithms by a large margin (6.7%, 2.8%, 5.2% respectively) on Stanford Dogs 120, Caltech-UCSD Birds 2011-200 and Caltech 256.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ayan Kumar Bhunia
2. Yi-Zhe Song
3. Xiu-Shen Wei
4. Jian Yang
5. Jianxin Wu
6. Jinhui Tang
7. Yuxin Peng
8. Zhanyu Ma

In Computer Vision and Pattern Recognition

1. Yi-Zhe Song
2. Luc Van Gool
3. Dacheng Tao
4. Tao A. Xiang
5. Yuxin Peng
6. Lei Zhang
7. Ayan Kumar Bhunia
8. Jian Yang

Citations in the corpus

• Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches (2020, methodology): “8 Stacked LSTM (CVPR19) [12] ResNet50 90.”
• Fine-Grained Image Analysis With Deep Learning: A Survey (2021, methodology): “LSTMs [71], [73], graphs [74], or knowledge distilling [75]) these joint part feature learning methods yield significantly”
• The Devil is in the Channels: Mutual-Channel Loss for Fine-Grained Image Classification (2020, unknown): “(3,3) WPS-CPM (CVPR19 [12]) GoogLeNet + ResNet50 90.”
• Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification (2021, unknown): “0 Stacked LSTM [13] 90.”