Cross-Domain Image Retrieval with a Dual Attribute-Aware Ranking Network

By Junshi Huang, Rogério Schmidt Feris, Qiang Chen, Shuicheng Yan. 2015

428 citations recorded by the source corpus.

We address the problem of cross-domain image retrieval, considering the following practical application: given a user photo depicting a clothing image, our goal is to retrieve the same or attribute-similar clothing items from online shopping stores. This is a challenging problem due to the large discrepancy between online shopping images, usually taken in ideal lighting/pose/background conditions, and user photos captured in uncontrolled conditions. To address this problem, we propose a Dual Attribute-aware Ranking Network (DARN) for retrieval feature learning. More specifically, DARN consists of two sub-networks, one for each domain, whose retrieval feature representations are driven by semantic attribute learning. We show that this attribute-guided learning is a key factor for retrieval accuracy improvement. In addition, to further align with the nature of the retrieval problem, we impose a triplet visual similarity constraint for learning to rank across the two subnetworks. Another contribution of our work is a large-scale dataset which makes the network learning feasible. We exploit customer review websites to crawl a large set of online shopping images and corresponding offline user photos with fine-grained clothing attributes, i.e., around 450,000 online shopping images and about 90,000 exact offline counterpart images of those online ones. All these images are collected from real-world consumer websites reflecting the diversity of the data modality, which makes this dataset unique and rare in the academic community. We extensively evaluate the retrieval performance of networks in different configurations. The top-20 retrieval accuracy is doubled when using the proposed DARN other than the current popular solution using pre-trained CNN features only (0.570 vs. 0.268).

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

Citations in the corpus

Revisiting IM2GPS in the Deep Learning Era (2017, background): “Image retrieval using learned, deep representations is useful to a wide range of tasks such as product ranking [4, 11], sketch based image retrieval [25], face recognition [31, 26], cross-view localization [18, 37, 34] and scene retrieval [33, 2, 23, 7].”
Pixel-Level Domain Transfer (2016, background): “different image domain has been proposed in computer vision [9,10,11,12,13,14], but all these adaptations take place in the feature space, i.”
Fully-Adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification (2016, background): “Multi-task networks have been used with base layers that are shared across all attributes, and branches to encode task-speciﬁc features for each attribute category [13, 37].”
DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Images (2019, methodology): “Several clothes datasets have been proposed such as [20, 5, 7, 14, 21, 1] as summarized in Table 1.”
Fashion Landmark Detection in the Wild (2016, background): “detecting the clothes bounding boxes [1,2] or the human joints [8,9].”
DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations (2016, methodology): “The previous datasets were labeled with limited number of attributes, bounding boxes [12], or consumer-to-shop pair correspondences [10].”
Localizing and Orienting Street Views Using Overhead Imagery (2016, background): “For example, ranking can be improved by adding a classiﬁcation layer predicting category [4, 25] or attributes [14].”
Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback (2021, methodology): “Descriptive representations such as attributes can often be extracted from this data, and form a strong basis for generating stronger image captions [71, 66, 70] and more effective image retrieval [25, 5, 51, 34].”
Imbalanced Deep Learning by Minority Class Incremental Rectification (2018, methodology): “These experimental results show a clear advantage of CRL over 12 state-of-the-art models compared, including 7 attribute models (PANDA [18], ANet [12], Triplet-kNN [39], FashionNet [30], DARN [29], LMLE [17], MTCT [37]).”
Stochastic Filter Groups for Multi-Task CNNs: Learning Specialist and Generalist Convolution Kernels (2019, background): “1 shows a typical CNN architecture for MTL comprised of a shared “trunk” feature extractor and task-specific “branch” networks [32, 9, 13, 15, 27, 2].”
Fine-Grained Image Retrieval: the Text/Sketch Input Dilemma (2017, methodology): “Sampling strategy Our quadruplet sampler is inspired by the sampling strategy proposed in [10].”
AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning (2019, background): “Hard-parameter sharing usually relies on hand-designed architectures composed of hidden layers that are shared across all tasks and specialized branches that learn task-specific features [23, 29, 43, 5, 26, 12].”
Automatic Spatially-Aware Fashion Concept Discovery (2017, background): “There have been several clothing datasets collected recently [16, 8, 21, 6, 7].”
Deep Multimodality Model for Multi-task Multi-view Learning (2019, neutral): “DARN [8] (DeepFashion data set); .”
Real-Time Online Action Detection Forests Using Spatio-Temporal Contexts (2016, background): “In particular, they have been demonstrated to be effective in capturing the ﬁne-grained attributes of human actions [18] (as well as other scene objects [17]) providing robustness to appearance variations [37, 52] often encountered in human action recognition.”