SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering

By Chenguang Zhu, Michael Zeng, Xuedong Huang. 2018

131 citations recorded by the source corpus.

Conversational question answering (CQA) is a novel QA task that requires understanding of dialogue context. Different from traditional single-turn machine reading comprehension (MRC) tasks, CQA includes passage comprehension, coreference resolution, and contextual understanding. In this paper, we propose an innovated contextualized attention-based deep neural network, SDNet, to fuse context into traditional MRC models. Our model leverages both inter-attention and self-attention to comprehend conversation context and extract relevant information from passage. Furthermore, we demonstrated a novel method to integrate the latest BERT contextual model. Empirical results show the effectiveness of our model, which sets the new state of the art result in CoQA leaderboard, outperforming the previous best model by 1.6% F1. Our ensemble model further improves the result by 2.7% F1.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Zhe Gan
3. Mohit Bansal
4. Jingjing Liu
5. Yu Cheng
6. Caiming Xiong
7. Luke S. Zettlemoyer
8. Furu Wei

In Computer Vision and Pattern Recognition

1. Zhe Gan
2. Jingjing Liu
3. Yu Cheng
4. Steven C. H. Hoi
5. Dacheng Tao
6. Jianfeng Gao
7. Luc Van Gool
8. Mohit Bansal

In Natural Language Processing

1. Jianfeng Gao
2. Zhe Gan
3. Jingjing Liu
4. Yu Cheng
5. Mohit Bansal
6. Christopher D. Manning
7. Siva Reddy
8. Danqi Chen

Citations in the corpus

• CoQA: A Conversational Question Answering Challenge (2018, positive): “Others (e.g., Zhu et al., 2018) attempted to incorporate the most recent pretrained language representation model BERT (Devlin et al., 2018)12 into CoQA and demonstrated superior results.”
• Patient Knowledge Distillation for BERT Model Compression (2019, methodology): “In some variants of BERT, like SDNet (Zhu et al., 2018), a weighted average of all layers’ [CLS] embeddings is applied.”
• Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems (2019, background): “…(Vaswani et al., 2017) for incorporating attention and focus on different dialogue settings, e.g. text-only or response selection settings (Zhu et al., 2018; Mazaré et al., 2018; Dinan et al., 2018), These approaches consider the knowledge to be grounded in text, whereas in VGDS, the…”
• Dialog State Tracking: A Neural Reading Comprehension Approach (2019, methodology): “…models into our dialog state tracking pipeline, a slot carryover model to help the tracker make a binary decision whether the slot values from the previous turn should be used; a slot type model to predict whether the answer is { Yes , No , DontCare , Span } , which is similar to Zhu et al. (2018).”
• Open-Retrieval Conversational Question Answering (2020, result): “Response Ranking w/ UDC [27, 49, 53] ✗ ✓ ✓ ✓ Conversational MC w/ CoQA [5, 61] ✗ ✓ ✗ ✗ Conversational MC w/ QuAC [18, 34] ✗ ✓ ✓ ✓ ORConvQA w/ OR-QuAC (this work) ✓ ✓ ✓ ✓”