Making History Matter: History-Advantage Sequence Training for Visual Dialog

By Tianhao Yang, Zhengjun Zha, Hanwang Zhang. 2019

72 citations recorded by the source corpus.

We study the multi-round response generation in visual dialog, where a response is generated according to a visually grounded conversational history. Given a triplet: an image, Q&A history, and current question, all the prevailing methods follow a codec (i.e., encoder-decoder) fashion in a supervised learning paradigm: a multimodal encoder encodes the triplet into a feature vector, which is then fed into the decoder for the current answer generation, supervised by the ground-truth. However, this conventional supervised learning does NOT take into account the impact of imperfect history, violating the conversational nature of visual dialog and thus making the codec more inclined to learn history bias but not contextual reasoning. To this end, inspired by the actor-critic policy gradient in reinforcement learning, we propose a novel training paradigm called History Advantage Sequence Training (HAST). Specifically, we intentionally impose wrong answers in the history, obtaining an adverse critic, and see how the historic error impacts the codec’s future behavior by History Advantage — a quantity obtained by subtracting the adverse critic from the gold reward of ground-truth history. Moreover, to make the codec more sensitive to the history, we propose a novel attention network called History-Aware Co-Attention Network (HACAN) which can be effectively trained by using HAST. Experimental results on three benchmarks: VisDial v0.9&v1.0 and GuessWhat?!, show that the proposed HAST strategy consistently outperforms the state-of-the-art supervised counterparts.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

In Dialogue & Conversational AI

Citations in the corpus

DualVD: An Adaptive Dual Encoding Model for Deep Visual Understanding in Visual Dialogue (2019, background): “Most approaches focus on enhancing the textural representations by recovering the dialogue relational structure (Zheng et al. 2019), imperfect dialogue history (Yang, Zha, and Zhang 2019), and dialogue consistency (Qi et al. 2018).”
DialGraph: Sparse Graph Learning Networks for Visual Dialog (2020, unknown): “81 HACAN [45] 57.”
Multi-View Attention Networks for Visual Dialog (2020, methodology): “In addition, other approaches proposed different training methods for the visual dialog task including reinforcement mechanism (Yang et al., 2019) and adversarial learning (Lu et al.”
Efficient Attention Mechanism for Handling All the Interactions between Many Inputs with Application to Visual Dialog (2019, methodology): “They can be categorized into the following three groups by the design of the encoder: i) fusion-based methods, e.g., LF [8] and HRE [8], which fuses the inputs by their concatenation followed by the application of a feed-forward or recurrent network, and Syn-ergistic [15], which fuses the inputs at ”