Image Retrieval Using Textual Cues
By Anand Mishra, Alahari Karteek, C. V. Jawahar. 2013
127 citations recorded by the source corpus.
We present an approach for the text-to-image retrieval problem based on textual content present in images. Given the recent developments in understanding text in images, an appealing approach to address this problem is to localize and recognize the text, and then query the database, as in a text retrieval problem. We show that such an approach, despite being based on state-of-the-art methods, is insufficient, and propose a method, where we do not rely on an exact localization and recognition pipeline. We take a query-driven search approach, where we find approximate locations of characters in the text query, and then impose spatial constraints to generate a ranked list of images in the database. The retrieval performance is evaluated on public scene text datasets as well as three large datasets, namely IIIT scene text retrieval, Sports-10K and TV series-1M, we introduce.
Subfields: Computer Vision and Pattern Recognition, Information Retrieval & Search, Multimodal Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Trevor Darrell
- Dacheng Tao
- Andrew Zisserman
- Xiaogang Wang
- Andrea Vedaldi
- Ming-Hsuan Yang
- Jitendra Malik
In Information Retrieval & Search
- Ali Furkan Biten
- Yijuan Lu
- Devi Parikh
- Andrea Vedaldi
- Andrew Zisserman
- C. V. Jawahar
- Dimosthenis Karatzas
- Dhruv Batra
In Multimodal Learning
- Lijuan Wang
- Devi Parikh
- Jianfeng Wang
- Ronghang Hu
- Zhe Gan
- Dhruv Batra
- Marcus Rohrbach
- Zhengyuan Yang
Citations in the corpus
- TAP: Text-Aware Pre-training for Text-VQA and Text-Caption (2020, methodology): “The ST-VQA dataset [8] contains 21,892 images from multiple sources including ICDAR 2013 [27], ICDAR 2015 [26], ImageNet [13], VizWiz [18], IIIT STR [39], Visual Genome [29], and COCO-Text [54].”
- Reading Text in the Wild with Convolutional Neural Networks (2014, background): “STR IIIT STR (Mishra et al. 2013) text retrieval dataset 50 10k”
- Scene Text Visual Question Answering (2019, methodology): “In total, we used six different datasets, namely: ICDAR 2013[27] and ICDAR2015[26], ImageNet [10], VizWiz[18], IIIT Scene Text Retrieval[42], Visual Genome [33] and COCO-Text [53].”
- LaTr: Layout-Aware Transformer for Scene-Text VQA (2021, methodology): “ST-VQA [8] is an amalgamation of well-known computer vision datasets, namely: ICDAR 2013 [25], IC-DAR2015 [24], ImageNet [10], VizWiz [18], IIIT Scene Text Retrieval [46], Visual Genome [28] and COCO-Text [64].”
- On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answering (2020, unknown): “[4] 19k 26k 3k 4k [6, 8, 13, 14, 17, 23, 32] [24] 180k 900k 20k 100k [9] [29] 25k 39k 3k 5k [16]”
- Spatially Aware Multimodal Transformers for TextVQA (2020, background): “ST-VQA contains 18,921 training and 2,971 test images sourced from several datasets [5, 16, 17, 20, 31].”
- Iterative Answer Prediction With Pointer-Augmented Multimodal Transformers for TextVQA (2019, methodology): “The ST-VQA dataset [8] contains natural images from multiple sources including ICDAR 2013 [24], ICDAR 2015 [23], ImageNet [12], VizWiz [18], IIIT STR [36], Visual Genome [26], and COCO-Text [49].”