Dual Visual Attention Network for Visual Dialog

By Dan Guo, Hui Wang, Meng Wang. 2019

36 citations recorded by the source corpus.

Visual dialog is a challenging task, which involves multi-round semantic transformations between vision and language. This paper aims to address cross-modal semantic correlation for visual dialog. Motivated by that Vg (global vision), Vl (local vision), Q (question) and H (history) have inseparable relevances, the paper proposes a novel Dual Visual Attention Network (DVAN) to realize (Vg, Vl, Q, H)--> A. DVAN is a three-stage query-adaptive attention model. In order to acquire accurate A (answer), it first explores the textual attention, which imposes the question on history to pick out related context H'. Then, based on Q and H', it implements respective visual attentions to discover related global image visual hints Vg' and local object-based visual hints Vl'. Next, a dual crossing visual attention is proposed. Vg' and Vl' are mutually embedded to learn the complementary of visual semantics. Finally, the attended textual and visual features are combined to infer the answer. Experimental results on the VisDial v0.9 and v1.0 datasets validate the effectiveness of the proposed approach.

Subfields: Computer Vision and Pattern Recognition, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Meng Wang
2. Can Xu
3. Daxin Jiang
4. Xiuyi Chen
5. Hanwang Zhang
6. Zhengjun Zha
7. Dan Guo
8. Haibo Wang

In Dialogue & Conversational AI

1. Xiuyi Chen
2. Can Xu
3. Daxin Jiang
4. Zhengjun Zha
5. Dan Guo
6. Hanwang Zhang

Citations in the corpus

• VD-BERT: A Unified Vision and Dialog Transformer with BERT (2020, neutral): “Discriminative Setting Generative Setting MRR ↑ R@1 ↑ R@5 ↑ R@10 ↑ Mean ↓ MRR ↑ R@1 ↑ R@5 ↑ R@10 ↑ Mean ↓ LF (Das et HACAN (Yang et al., 2019), Synergistic (Guo et al., 2019a), DAN (Kang et al., 2019), and ReDAN (Gan et al., 2019).”
• Iterative Context-Aware Graph Inference for Visual Dialog (2020, methodology): “To improve performance, various attention mechanisms have been widely used in the task, including history-conditioned image attention (HCIAE) [23], sequential co-attention (CoAtt) [34], dual visual attention (DVAN) [9], and recurrent dual attention (ReDAN) [7].”
• Learning to Ground Visual Objects for Visual Dialog (2021, methodology): “, 2019), DVAN (Guo et al., 2019b) and DMRM (Chen et al.”