ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems
By Qi Zhu, Zheng Zhang, Yan Fang, Xiang Li, Ryuichi Takanobu, Jinchao Li, Baolin Peng, Jianfeng Gao, Xiaoyan Zhu, Minlie Huang. 2020
118 citations recorded by the source corpus.
We present ConvLab-2, an open-source toolkit that enables researchers to build task-oriented dialogue systems with state-of-the-art models, perform an end-to-end evaluation, and diagnose the weakness of systems. As the successor of ConvLab, ConvLab-2 inherits ConvLab’s framework but integrates more powerful dialogue models and supports more datasets. Besides, we have developed an analysis tool and an interactive tool to assist researchers in diagnosing dialogue systems. The analysis tool presents rich statistics and summarizes common mistakes from simulated dialogues, which facilitates error analysis and system improvement. The interactive tool provides an user interface that allows developers to diagnose an assembled dialogue system by interacting with the system and modifying the output of each system component.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Chunyuan Li
- Jianfeng Gao
- Baolin Peng
- Michael Heck
- Carel van Niekerk
- Nurul Lubis
- Milica Gavsi'c
- Christian Geishauser
Citations in the corpus
- Dynamic Dialogue Policy for Continual Reinforcement Learning (2022, methodology): “The approaches directly vectorise the belief state, hence the size of the input vector depends on the domain (as different domains for instance have different numbers of slots) (Su et al., 2016; Lipton et al., 2018; Weisz et al., 2018; Takanobu et al., 2019; Zhu et al., 2020).”
- Domain-independent User Simulation with Transformers for Task-oriented Dialogue Systems (2021, methodology): “, 2017), a simple and stable reinforcement learning algorithm, with ABUS, VHUS, and TUS as USs in the ConvLab-2 framework (Zhu et al., 2020).”
- Uncertainty Measures in Neural Belief Tracking and the Effects on Dialogue Policy Performance (2021, methodology): “We incorporate SetSUMBT, EnD-SetSUMBT and EnD 2 -SetSUMBT within the Convlab2 (Zhu et al., 2020) task-oriented dialogue environment and compare their performance by training policies which take belief states as inputs 4 .”
- SOLOIST: Few-shot Task-Oriented Dialog with A Single Pre-trained Auto-regressive Model (2020, methodology): “For example, recent attempts such as RASA (Bocklisch et al., 2017), ConvLab (Lee et al., 2019; Zhu et al., 2020), and Conversation Learner (Shukla et al., 2020) have been made to allow the use of data-driven approaches based on machine learning or machine teaching for the development of dialog…”
- Dynamic Dialogue Policy Transformer for Continual Reinforcement Learning (2022, methodology): “We use ConvLab-2 (Zhu et al., 2020) as the back-bone of our implementation.”