LSDSCC: a Large Scale Domain-Specific Conversational Corpus for Response Generation with Diversity Oriented Evaluation Metrics
By Zhen Xu, Nan Jiang, Bingquan Liu, Wenge Rong, Bowen Wu, Baoxun Wang, Zhuoran Wang, Xiaolong Wang. 2018
14 citations recorded by the source corpus.
It has been proven that automatic conversational agents can be built up using the Endto-End Neural Response Generation (NRG) framework, and such a data-driven methodology requires a large number of dialog pairs for model training and reasonable evaluation metrics for testing. This paper proposes a Large Scale Domain-Specific Conversational Corpus (LSDSCC) composed of high-quality queryresponse pairs extracted from the domainspecific online forum, with thorough preprocessing and cleansing procedures. Also, a testing set, including multiple diverse responses annotated for each query, is constructed, and on this basis, the metrics for measuring the diversity of generated results are further presented. We evaluate the performances of neural dialog models with the widely applied diversity boosting strategies on the proposed dataset. The experimental results have shown that our proposed corpus can be taken as a new benchmark dataset for the NRG task, and the presented metrics are promising to guide the optimization of NRG models by quantifying the diversity of the generated responses reasonably.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jason Aaron Edward Weston
- Minh-Thang Luong
- David R. So
- Quoc V. Le
- Mohit Bansal
- Yifeng Lu
- Xiaolin Li
- Jianfeng Gao
In Natural Language Processing
- Minh-Thang Luong
- Jason Aaron Edward Weston
- Quoc V. Le
- David R. So
- Xiaolin Li
- Qile Zhu
- Mohit Bansal
- Xiyao Ma
Citations in the corpus
- Towards a Human-like Open-Domain Chatbot (2020, background): “Many works, e.g., Zhao et al. (2017); Xu et al. (2018); Ippolito et al. (2019b), have focused solely on the diversity aspect to counter the commonly observed problem of models generating generic responses (Li et al., 2016a).”
- Improving Question Generation with Sentence-level Semantic Matching and Answer Position Inferring (2019, background): “Auxiliary Metrics Although the main metrics can reflect the similarity between the generated question and the references, it has its limits on reflecting the semantics of generated question (Xu et al. 2018).”