Fine-Grained Image Classification by Exploring Bipartite-Graph Labels

By Feng Zhou, Yuanqing Lin. 2015

139 citations recorded by the source corpus.

Given a food image, can a fine-grained object recognition engine tell "which restaurant which dish" the food belongs to? Such ultra-fine grained image recognition is the key for many applications like search by images, but it is very challenging because it needs to discern subtle difference between classes while dealing with the scarcity of training data. Fortunately, the ultra-fine granularity naturally brings rich relationships among object classes. This paper proposes a novel approach to exploit the rich relationships through bipartite-graph labels (BGL). We show how to model BGL in an overall convolutional neural networks and the resulting system can be optimized through back-propagation. We also show that it is computationally efficient in inference thanks to the bipartite structure. To facilitate the study, we construct a new food benchmark dataset, which consists of 37,885 food images collected from 6 restaurants and totally 975 menus. Experimental results on this new food and three other datasets demonstrate BGL advances previous works in fine-grained object recognition. An online demo is available at http: //www.f-zhou.com/fg_demo/.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

SpatialFlow: Bridging All Tasks for Panoptic Segmentation (2019, background): “These methods have dramatically improved the state-of-the-art in many vision tasks, including image recognition[23], [25], [26], [27], [28], [29], [30], image retrieval [31], [32], metric learning [33], [34], [35], object detection [36], [37], [19], image segmentation [38], [39], [4], human pose est”
Answering Visual-Relational Queries in Web-Extracted Knowledge Graphs (2017, methodology): “[Zhou and Lin, 2016] propose a method based on a bipartite graph that links depictions of meals to its ingredients.”
A Survey on Food Computing (2018, background): “[Zhang et al. 2016a] proposed a multi-task system that can identify dish types, food ingredients and cooking methods from food images with CNN. [Zhou and Lin 2016] proposed a novel approach to exploit rich ingredient and label relationships through bipartite-graph labels, and then combined…”
Learning a Discriminative Filter Bank Within a CNN for Fine-Grained Recognition (2016, background): “Others have explored directions such as utilizing hierarchical label structures [57], combining visual and textual information [54, 2, 16], 3D-assisted recognition [26, 31, 37], introducing humans in the loop [40, 4, 7], and collecting larger amount of data [45, 47, 25, 17].”
Knowledge Concentration: Learning 100K Object Classifiers in a Single CNN (2017, background): “[26] exploited the rich relationships through bipartite-graph labels.”
Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition (2018, background): “For instance, the bipartite-graph labeling [23] leverages the label hierarchy on the fine-grained classes, which is less expensive to obtain.”
Object-Part Attention Model for Fine-Grained Image Classification (2017, methodology): “Most existing works rely on object or part annotations [6], [12], [13], while labeling is heavily labor consuming.”
Fine-Grained Image Analysis With Deep Learning: A Survey (2021, methodology): “BGL [50] was proposed to incorporate rich bipartitegraph labels into CNN training to model the important relationships among fine-grained classes.”
Weakly Supervised Learning of Part Selection Model with Spatial Constraints for Fine-Grained Image Classification (2017, methodology): “When applying CNN model, our method is the best among methods under the same setting (Zhang et al. 2016b; Simon and Rodner 2015; Xiao et al. 2015; Zhou and Lin 2015), and obtains a 2.09% higher accuracy than the best performing result of PD (Zhang et al. 2016b) (82.60%).”