Automatic Spatially-Aware Fashion Concept Discovery

By Xintong Han, Zuxuan Wu, Phoenix X. Huang, Xiao Zhang, Menglong Zhu, Yuan Li, Yang Zhao, Larry S. Davis. IEEE International Conference on Computer Vision, 2017

326 citations recorded by the source corpus.

This paper proposes an automatic spatially-aware concept discovery approach using weakly labeled image-text data from shopping websites. We first fine-tune GoogleNet by jointly modeling clothing images and their corresponding descriptions in a visual-semantic embedding space. Then, for each attribute (word), we generate its spatiallyaware representation by combining its semantic word vector representation with its spatial representation derived from the convolutional maps of the fine-tuned network. The resulting spatially-aware representations are further used to cluster attributes into multiple groups to form spatiallyaware concepts (e.g., the neckline concept might consist of attributes like v-neck, round-neck, etc). Finally, we decompose the visual-semantic embedding space into multiple concept-specific subspaces, which facilitates structured browsing and attribute-feedback product retrieval by exploiting multimodal linguistic regularities. We conducted extensive experiments on our newly collected Fashion200K dataset, and results on clustering quality evaluation and attribute-feedback product retrieval task demonstrate the effectiveness of our automatically discovered spatially-aware concepts.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/782188821963304fb78791e01665590f0cd869e8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models (2021, background): “However, existing datasets are constrained to domains such as fashion products [4, 12, 13] or synthetic objects [40] with relatively simple image contents.”
Learning Fashion Compatibility with Bidirectional LSTMs (2017, background): “However, most of them focus on clothing parsing [10, 28], clothing recognition [13], or clothing retrieval [6, 11].”
VITON: An Image-Based Virtual Try-on Network (2017, background): “, color and textures) of a clothing item [25, 48, 15], since virtual try-on requires preserving the details of a target clothing image as much as possible, including exactly the same style, embroidery, logo, text, etc.”
RetrieveGAN: Image Synthesis via Differentiable Patch Retrieval (2020, background): “Image retrieval has been a classical vision problem with numerous applications such as product search [24,6,1], multimodal image retrieval [36,3], image geolocalization [7], event detection [13], among others.”
On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey (2025, unknown): “Text Pairs ( 𝑥, 𝑦 + / 𝑦 − ) Text Triples ( 𝑥, 𝑦 + , 𝑦 − ) Instructed Triples ( 𝐼 ◦ 𝑥, 𝑦 + , 𝑦 − ) mistral”