MeLL: Large-scale Extensible User Intent Classification for Dialogue Systems with Meta Lifelong Learning

By Chengyu Wang, Haojie Pan, Yuan Liu, Kehan Chen, Minghui Qiu, Wei Zhou, Jun Huang, Haiqing Chen, Wei Lin, Deng Cai. 2021

33 citations recorded by the source corpus.

User intent detection is vital for understanding their demands in dialogue systems. Although the User Intent Classification (UIC) task has been widely studied, for large-scale industrial applications, the task is still challenging. This is because user inputs in distinct domains may have different text distributions and target intent sets. When the underlying application evolves, new UIC tasks continuously emerge in a large quantity. Hence, it is crucial to develop a framework for large-scale extensible UIC that continuously fits new tasks and avoids catastrophic forgetting with an acceptable parameter growth rate. In this paper, we introduce the Meta Lifelong Learning (MeLL) framework to address this task. In MeLL, a BERT-based text encoder is employed to learn robust text representations across tasks, which is slowly updated for lifelong learning. We design global and local memory networks to capture the cross-task prototype representations of different classes, treated as the meta-learner quickly adapted to different tasks. Additionally, the Least Recently Used replacement policy is applied to manage the global memory such that the model size does not explode through time. Finally, each UIC task has its own task-specific output layer, with the attentive summarization of various features. We have conducted extensive experiments on both open-source and real industry datasets. Results show that MeLL improves the performance compared with strong baselines and also reduces the number of total parameters. We have also deployed MeLL on a real-world e-commerce dialogue system AliMe and observed significant improvements in terms of both F1 and the resources usage.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Dialogue & Conversational AI, Meta & Continual Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Meta & Continual Learning

Citations in the corpus

Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022, background): “Sun et al. (2020a); Wang et al. (2021); Madotto et al. (2021) formulate multiclass-text classification as a task-incremental learning ( Task-IL ) problem where the objective is to train an “incremental learner” on a sequence of tasks.”
TransPrompt: Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification (2021, unknown): “Similar to Wang et al. (2020a); Pan et al. (2021); Wang et al. (2021), the proposed TransPrompt framework can be viewed as a combination of transfer learning and meta-learning, which learns transferable knowledge from similar tasks to improve the performance of few-shot text classiﬁcation, either…”
Continual Few-shot Intent Detection (2022, background): “Recently, Liu et al. (2021) and Wang et al. (2021) have made some efforts on the Continual Intent Detection task (CID).”