Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling

By Bing Liu, Ian Lane. 2016

721 citations recorded by the source corpus.

Attention-based encoder-decoder neural network models have recently shown promising results in machine translation and speech recognition. In this work, we propose an attention-based neural network model for joint intent detection and slot filling, both of which are critical steps for many speech understanding and dialog systems. Unlike in machine translation and speech recognition, alignment is explicit in slot filling. We explore different strategies in incorporating this alignment information to the encoder-decoder framework. Learning from the attention mechanism in encoder-decoder model, we further propose introducing attention to the alignment-based RNN models. Such attentions provide additional information to the intent classification and slot label prediction. Our independent task models achieve state-of-the-art intent detection error rate and slot filling F1 score on the benchmark ATIS task. Our joint training model further obtains 0.56% absolute (23.8% relative) error reduction on intent detection and 0.23% absolute gain on slot filling over the independent task models.

Subfields: Artificial Intelligence, Natural Language Processing, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Jianfeng Gao
  2. Bing Liu
  3. Mohit Bansal
  4. Luke S. Zettlemoyer
  5. Richard Socher
  6. Pascale Fung
  7. Percy Liang
  8. Erik Cambria

In Natural Language Processing

  1. Jianfeng Gao
  2. Pascale Fung
  3. Erik Cambria
  4. Bing Liu
  5. Andrea Madotto
  6. Ting-Wei Liu
  7. Philip S. Yu
  8. Mohit Bansal

In Information Extraction

  1. Hongxia Jin
  2. Yilin Shen
  3. Serge Sharoff
  4. Yu Wang
  5. Bing Liu
  6. Ian Lane
  7. Tim Weninger
  8. Abhishek Patel

Citations in the corpus