History for Visual Dialog: Do we really need it?

By Shubham Agarwal, Trung Bui, Joon-Young Lee, Ioannis Konstas, Verena Rieser. 2020

77 citations recorded by the source corpus.

Visual Dialogue involves “understanding” the dialogue history (what has been discussed previously) and the current question (what is asked), in addition to grounding information in the image, to accurately generate the correct response. In this paper, we show that co-attention models which explicitly encode dialoh history outperform models that don’t, achieving state-of-the-art performance (72 % NDCG on val set). However, we also expose shortcomings of the crowdsourcing dataset collection procedure, by showing that dialogue history is indeed only required for a small amount of the data, and that the current evaluation metric encourages generic replies. To that end, we propose a challenging subset (VisdialConv) of the VisdialVal set and the benchmark NDCG of 63%.

Subfields: Computer Vision and Pattern Recognition, Multimodal Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Multimodal Learning

Citations in the corpus

Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, background): “…mind in conversational AI systems being solvable without extracting any information about agents (extensively discussed in Le et al. (2019)), or achieving state-of-the-art results on visual dialog without the need to consider dialog history and thus rendering it as VQA task Agarwal et al. (2020).”
VD-BERT: A Unified Vision and Dialog Transformer with BERT (2020, methodology): “There are recent works (Nguyen et al., 2019; Agarwal et al., 2020) also applying the Transformer to model the interactions among many entities.”
Multi-View Attention Networks for Visual Dialog (2020, background): “(2020) proposed causal intervention algorithms that can be applicable to other visual dialog models and Agarwal et al. (2020) proposed curriculum fine-tuning inspired by the work of Bengio et al.”
Learning to Ground Visual Objects for Visual Dialog (2021, methodology): “Agarwal et al., 2020; Chen et al., 2021b; Qi et al., 2020).”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, unknown): “However, Agarwal et al. (2020) argued that though with dialogue history the visual dialogue model could achieve better results, in fact only a small proportion of cases benefited from the history.”