Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention

By Wenhu Chen, Jianshu Chen, Pengda Qin, Xifeng Yan, William Yang Wang. 2019

133 citations recorded by the source corpus.

Semantically controlled neural response generation on limited-domain has achieved great performance. However, moving towards multi-domain large-scale scenarios is shown to be difficult because the possible combinations of semantic inputs grow exponentially with the number of domains. To alleviate such scalability issue, we exploit the structure of dialog acts to build a multi-layer hierarchical graph, where each act is represented as a root-to-leaf route on the graph. Then, we incorporate such graph structure prior as an inductive bias to build a hierarchical disentangled self-attention network, where we disentangle attention heads to model designated nodes on the dialog act graph. By activating different (disentangled) heads at each layer, combinatorially many dialog act semantics can be modeled to control the neural response generation. On the largescale Multi-Domain-WOZ dataset, our algorithm can yield an improvement of over 5.0 BLEU score, and in human evaluation, it also significantly outperforms other baselines over various metrics including consistency, etc.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

A Simple Language Model for Task-Oriented Dialogue (2020, methodology): “Decoding Initialized from pre-trained weights, SimpleTOD does not need to employ an advanced, more costly decoding strategy such as beam search, diverse beam search, and top-k sampling as opposed to HDSA [9] and DAMD [60].”
Few-shot Natural Language Generation for Task-Oriented Dialog (2020, methodology): “On F EW S HOT - WOZ, SC-GPT outperforms several strong baselines such as SC-LSTM and HDSA (Chen et al., 2019), showing that SC-GPT adapts to new domain much more effectively, requiring much smaller amounts of in-domain labels.”
Text-to-Text Pre-Training for Data-to-Text Tasks (2020, background): “(2020) and Chen et al. (2019b) show that pre-training in the form of GPT-2 can indeed improve performance on data-to-text task as well.”
Few-Shot Natural Language Generation by Rewriting Templates (2020, methodology): “Sequence to sequence architectures directly converting a sequential representation of system actions to system response are also very common (Wen et al., 2015; Dušek and Jurcicek, 2016b; Zhu et al., 2019; Chen et al., 2019a).”
MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems (2020, background): “Several follow up works (Wu et al., 2019a; Chen et al., 2019; Budzianowski and Vuli´c, 2019; Mehri et al., 2019; Madotto et al., 2020b) improved on the initial baselines with various methodologies.”
End-to-End Neural Pipeline for Goal-Oriented Dialogue Systems using GPT-2 (2020, background): “, 2019a) which maps the dialogue history to the dialogue state (the composite function of NLU and DST), and the Word-level POL (Budzianowski et al., 2018; Pei et al., 2019; Chen et al., 2019; Mehri et al., 2019; Zhao et al., 2019) which maps the previous utterance and dialogue state to the system re”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, methodology): “Task-oriented Dialogue Chen et al. (2019a) Court Debate Dataset A task-oriented dataset in judicial field containing court debate conversations.”
Template Guided Text Generation for Task Oriented Dialogue (2020, background): “Sequence to sequence architectures directly converting a sequential representation of system ac- tions to a system response are also very common (Wen et al., 2015; Du sek and Jurcicek, 2016b; Zhu et al., 2019; Chen et al., 2019).”
SOLOIST: Few-shot Task-Oriented Dialog with A Single Pre-trained Auto-regressive Model (2020, methodology): “(v) HDSA (Chen et al., 2019) is a modular dialog system which generates responses using BERT-based dialog policy and graph structure dialog act representations.”
Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models (2019, methodology): “, 2018), the HDSA (Chen et al., 2019) model that incorporates dialog act supervision as an inductive prior for model architecture, and the LaRL (Zhao et al.”
ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems (2020, methodology): “, 2018a), HDSA (Chen et al., 2019), and LaRL (Zhao et al.”
Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems (2020, background): “The current state-of-the-arts (Lei et al., 2018; Zhang et al., 2019a; Mehri et al., 2019; Chen et al., 2019; Peng et al., 2020a; Hosseini-Asl et al., 2020) are end-to-end pipelined systems that rely on Dialogue State Tracking (DST) and Speech Act (S-ACT) annotations.”
Structured Fusion Networks for Dialog (2019, methodology): “SFNs are outperformed by the recently published HDSA (Chen et al., 2019) models which relies on BERT (Devlin et al., 2018) and conditioning on graph structured dialog acts.”