Referring Expression Generation and Comprehension via Attributes

By Jingyu Liu, Liang Wang, Ming-Hsuan Yang. 2017

169 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Question Answering & Reading Comprehension, Natural Language Generation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dacheng Tao
2. Qi Wu
3. Hanwang Zhang
4. Zhe Gan
5. Anton van den Hengel
6. Mohit Bansal
7. Chunhua Shen
8. Zhou Yu

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Luc Van Gool
3. Yi Yang
4. Xiaogang Wang
5. Chunhua Shen
6. Ming-Hsuan Yang
7. Qi Tian
8. Cordelia Schmid

In Question Answering & Reading Comprehension

1. Anton van den Hengel
2. Zhou Yu
3. Qi Wu
4. Jun Yu
5. Chunhua Shen
6. Dacheng Tao
7. Hanwang Zhang
8. Peter Anderson

In Natural Language Generation

1. Jiebo Luo
2. Xudong Jiang
3. Chang Liu
4. Henghui Ding
5. Liang Wang
6. Qi Wu
7. Marimuthu Kalimuthu
8. William Yang Wang

Citations in the corpus

• DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding (2022, methodology): “The top-down methods (Hong et al. 2020; Hu et al. 2016; Liu, Wang, and Yang 2017; Liu et al. 2018; Nagaraja, Morariu, and Davis 2016; Mao et al. 2015) view the task as a ranking task with a set of boxes predicted by a pre-trained object detector, which limits their performance.”
• Learning to Assemble Neural Module Tree Networks for Visual Grounding (2018, methodology): “According to whether the model requires language composition, we group those methods into: 1) Generation based methods which select the region with the maximum generation probability: MMI [26], Attribute [21], and Listener [39].”
• Learning Semantic Concepts and Order for Image and Sentence Matching (2017, background): “To comprehensively predict all the semantic concepts for the image, a possible way is to adaptively explore the attribute learning frameworks [6, 21, 35, 37].”
• Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods (2019, background): “Most of the above approaches use bounding box localization, additionally object segmentation (Liu et al., 2017) is also explored for referring expression comprehension.”
• More Grounded Image Captioning by Distilling Image-Text Matching Model (2020, background): “Many works [39, 32, 5, 17, 62, 30] belong to the first category which requires expensive ground truth annotations.”
• Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic (2023, background): “We collection training data from public VQA, Image Captioning datset, and several datasets already containing positional annotation, such as Re-fCOCO (Kazemzadeh et al., 2014) for REC/REG, visual gemone (Krishna et al., 2017) for grounding caption, Visual-7W (Mani et al., 2020) for PointQA.”
• REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments (2019, background): “Recent work casts this task as looking for the object that can generate its paired expressions [13, 18, 30] or jointly embedding the image and expression for matching estimation [6, 12, 16, 29].”
• CogVLM: Visual Expert for Pretrained Language Models (2023, unknown): “We evaluate our model on the typical datasets, including Visual7w [Zhu et al., 2016], RefCOCO [Liu et al., 2017], RefCOCO+, and RefCOCOg to ensure completeness.”
• GRES: Generalized Referring Expression Segmentation (2023, methodology): “The majority of RES methods are one-stage, while two-stage methods are more prevalent in REC [12, 14, 28, 33, 48, 50].”
• Multi-Task Collaborative Network for Joint Referring Expression Comprehension and Segmentation (2020, methodology): “Referring Expression Comprehension (REC) [11, 12, 19, 21, 44, 45, 48, 42, 37] and Referring Expression Segmentation (RES) [32, 16, 40, 25, 34] are two emerging tasks, which involves identifying the target visual instances according to a given linguistic expression.”
• Multimodal Unified Attention Networks for Vision-and-Language Interactions (2019, methodology): “te the used detection models with VGG-16 [67] or ResNet-101 [38] backbones. Method Proposal Generator RefCOCO RefCOCO+ RefCOCOg Dataset Detector Backbone TestA TestB Val TestA TestB Val Test Val Attr [68] COCO FRCN VGG-16 72.0 57.3 - 58.0 46.2 - - - CMN [69] COCO FRCN VGG-16 71.0 65.8 - 54.3 47.8 - ”
• Improving One-stage Visual Grounding by Recursive Sub-query Construction (2020, background): “Under the two-stage framework, various works explore different aspects to improve visual grounding, such as better exploiting attributes [25,48,27], object relationships [42,45,46,24], phrase co-occurrences [4,9,1], etc .”