Making History Matter: Gold-Critic Sequence Training for Visual Dialog

By Tianhao Yang, Zhengjun Zha, Hanwang Zhang. 2019

8 citations recorded by the source corpus.

We study the multi-round response generation in visual dialog systems, where a response is generated according to a visually grounded conversational history. Given a triplet: an image, Q&A history, and current question, all the prevailing methods follow a codec (ie, encoder-decoder) fashion in the supervised learning paradigm: a multimodal encoder encodes the triplet into a feature vector, which is then fed into the decoder for the current answer generation, supervised by the ground-truth answer. However, this conventional supervised learning does not take into account the impact of imperfect history in the codec training, violating the conversational nature of visual dialog and thus making the codec more inclined to learn dataset bias but not visual reasoning. To this end, inspired by the actor-critic policy gradient in reinforcement learning, we propose a novel training paradigm called Gold-Critic Sequence Training (GCST). Specifically, we intentionally impose wrong answers in the history, obtaining an adverse reward, and see how the historic error impacts the codec's future behavior by subtracting the gold-critic baseline --- reward obtained by using ground-truth history --- from the adverse reward. Moreover, to make the codec more sensitive to the history, we propose a novel attention network called Recurrent Co-Attention Network (RCAN) which can be effectively trained by using GCST. Experimental results on three benchmarks: VisDial0.9&1.0 and GuessWhat?!, show that the proposed GCST strategy consistently outperforms over state-of-the-art supervised counterparts under all metrics.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhe Gan
2. Jingjing Liu
3. Linjie Li
4. Jianfeng Gao
5. Yu Cheng
6. Mohit Bansal
7. Hanwang Zhang
8. Yejin Choi

In Computer Vision and Pattern Recognition

1. Zhe Gan
2. Linjie Li
3. Hanwang Zhang
4. Jianfeng Gao
5. Jingjing Liu
6. Dacheng Tao
7. Luc Van Gool
8. Yu Cheng

In Multimodal Learning

1. Zhe Gan
2. Linjie Li
3. Jingjing Liu
4. Jianfeng Gao
5. Yu Cheng
6. Hanwang Zhang
7. Lijuan Wang
8. Qi Wu

In Dialogue & Conversational AI

1. Hanwang Zhang
2. Verena Rieser
3. Jianfeng Gao

Citations in the corpus

• Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog (2019, background): “97 HACAN (Yang et al., 2019) 57.”
• Two Causal Principles for Improving Visual Dialog (2019, background): “Details can be found in most visual dialog methods [21, 41, 43].”
• DualVD: An Adaptive Dual Encoding Model for Deep Visual Understanding in Visual Dialogue (2019, background): “Most approaches focus on enhancing the textural representations by recovering the dialogue relational structure (Zheng et al. 2019), imperfect dialogue history (Yang, Zha, and Zhang 2019), and dialogue consistency (Qi et al. 2018).”
• History for Visual Dialog: Do we really need it? (2020, background): “However, there is very limited evidence that dialog history indeed matters for these tasks (Yang et al., 2019).”