Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback

By Xiaoxiao Guo, Hui Wu, Yupeng Gao, Steven J. Rennie, Rogério Schmidt Feris. Computer Vision and Pattern Recognition, 2021

430 citations recorded by the source corpus.

Conversational interfaces for the detail-oriented retail fashion domain are more natural, expressive, and user friendly than classical keyword-based search interfaces. In this paper, we introduce the Fashion IQ dataset to support and advance research on interactive fashion image retrieval. Fashion IQ is the first fashion dataset to provide human-generated captions that distinguish similar pairs of garment images together with side-information consisting of real-world product descriptions and derived visual attribute labels for these images. We provide a detailed analysis of the characteristics of the Fashion IQ data, and present a transformer-based user simulator and interactive image retriever that can seamlessly integrate visual attributes with image features, user feedback, and dialog history, leading to improved performance over the state of the art in dialog-based image retrieval. We believe that our dataset will encourage further work on developing more natural and real-world applicable conversational shopping assistants. 1

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/4b8be374f863a47aa11dd8acf1aeed9bf9dda402

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Text as Neural Operator:Image Manipulation by Text Instruction (2020, positive): “The task studied in this paper is inspired by cross-modal image retrieval – a cornerstone in many image retrieval tasks such as product search [6, 7, 16, 17, 32, 68, 80].”
Multimodal Learning With Transformers: A Survey (2022, background): “MULTIMODALQA (MMQA) [131], VALUE [132], Fashion IQ [133], LRS2-BBC [134], ActivityNet [135], VisDial [136].”
Towards Text-Image Interleaved Retrieval (2025, unknown): “…retrieval (MIR) aims to retrieve relevant information involving multiple modalities (Wei et al., 2024), which plays a crucial role in various applications such as e-commerce search (Wu et al., 2021) and retrieval augmented generation (RAG) systems (Chen et al., 2022; Ya-sunaga et al., 2023).”