EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training

By Hao Zhou, Pei Ke, Zheng Zhang, Yuxian Gu, Yinhe Zheng, Chujie Zheng, Yida Wang, Chen Henry Wu, Hao Sun, Xiaocong Yang and 4 more. 2021

58 citations recorded by the source corpus.

Although pre-trained language models have remarkably enhanced the generation ability of dialogue systems, open-domain Chinese dialogue systems are still limited by the dialogue data and the model size compared with English ones. In this paper, we propose EVA, a Chinese dialogue system that contains the largest Chinese pre-trained dialogue model with 2.8B parameters. To build this model, we collect the largest Chinese dialogue dataset named WDC-Dialogue from various public social media. This dataset contains 1.4B context-response pairs and is used as the pre-training corpus of EVA. Extensive experiments on automatic and human evaluation show that EVA outperforms other Chinese pre-trained dialogue models especially in the multi-turn interaction of human-bot conversations.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Minlie Huang
2. Xiaoyan Zhu
3. Chujie Zheng
4. Qun Liu
5. Zhexin Zhang
6. Hao Sun
7. Helen Mei-Ling Meng
8. Deng Jiawen

Citations in the corpus

• Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation (2022, background): “(3) Data curation, which usually filters offensive content in the training data using classifiers or set generic responses to the toxic contexts (Zhou et al., 2021; Gu et al., 2022; Mi et al., 2022).”
• EVA2.0: Investigating Open-domain Chinese Dialogue Systems with Large-scale Pre-training (2022, methodology): “Recently, large-scale pre-training [1, 2] has become a mainstream approach to building open-domain dialogue systems, both in English [3–5] and Chinese [6– 8].”
• CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models (2023, methodology): “To ensure data quality and diversity, the training data are derived from the filtered WDC-Dialogues (Zhou et al., 2021) dataset as well as publicly available datasets (Lison and Tiedemann, 2016; Guan et al., 2https://www.reddit.com/ 2021; Wu et al., 2019; Zhou et al., 2020a; Liu et al., 2020c; Wang ”
• COLD: A Benchmark for Chinese Offensive Language Detection (2022, methodology): “, 2021b), and EVA (Zhou et al., 2021), to investigate their strengths and weaknesses in terms of safety.”
• Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark (2022, background): “In recent years, significant efforts have been devoted to the development of open-domain dialog systems that are pre-trained on large-scale data to generate responses to user inputs (Freitas et al., 2020; Zhou et al., 2021a; Bao et al., 2021; Thop-pilan et al., 2022; Mi et al., 2022).”