Technical report on Conversational Question Answering

By Yingnan Ju, Fubang Zhao, Shijie Chen, Bowen Zheng, Xuefeng Yang, Yunfeng Liu. 2019

50 citations recorded by the source corpus.

Conversational Question Answering is a challenging task since it requires understanding of conversational history. In this project, we propose a new system RoBERTa + AT +KD, which involves rationale tagging multi-task, adversarial training, knowledge distillation and a linguistic post-process strategy. Our single model achieves 90.4(F1) on the CoQA test set without data augmentation, outperforming the current state-of-the-art single model by 2.6% F1.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Weizhu Chen
3. Xiaodong Liu
4. Pengcheng He
5. Nan Duan
6. Yelong Shen
7. Zhe Gan
8. Mohit Bansal

In Natural Language Processing

1. Jianfeng Gao
2. Xiaodong Liu
3. Weizhu Chen
4. Pengcheng He
5. Nan Duan
6. Hao Cheng
7. Yelong Shen
8. Wei Wang

In Evaluation & Benchmarks

1. Chenliang Li
2. Yu Yan
3. Yelong Shen
4. Jianfeng Gao
5. Jie Fu
6. Jiusheng Chen
7. Xiaodong Liu
8. Weizhu Chen

Citations in the corpus

• Pre-trained models for natural language processing: A survey (2020, neutral): “[78] proposed a “PTM+Adversarial Training+Rationale Tagging+Knowledge Distillation” model.”
• UnitedQA: A Hybrid Approach for Open Domain Question Answering (2021, background): “…first, we add a group of attention bias parameters into the decoder cross-attention block to feature the ranking information of retrieved contexts; second, we add the adversarial training (Ju et al., 2019; Jiang et al., 2020; Pereira et al., 2021) to improve the model generalization ability.”
• RikiNet: Reading Wikipedia Pages for Natural Question Answering (2020, background): “With the progress of large-scale pre-trained language models (Devlin et al., 2018), state-of-the-art MRC models (Ju et al., 2019; Yang et al., 2019; Lan et al., 2019; Zhang et al., 2019; Liu et al., 2019) have already surpassed human-level performance on certain commonly used MRC benchmark…”
• Addressing Semantic Drift in Generative Question Answering with Auxiliary Extraction (2021, background): “We then calculate the averaged cross entropy, similar to(Ju et al., 2019), for the rationale extraction loss: (2) where N is the number of input tokens. y ri is the rationale label for the i th token, and L RE j represents the rationale loss for the j th example in the training set.”