Recurrent Neural Networks with External Memory for Spoken Language Understanding

By Baolin Peng, Kaisheng Yao, Jing Li, Kam-Fai Wong. 2015

49 citations recorded by the source corpus.

Recurrent Neural Networks RNNs have become increasingly popular for the task of language understanding. In this task, a semantic tagger is deployed to associate a semantic label to each word in an input sequence. The success of RNN may be attributed to its ability to memorise long-term dependence that relates the current-time semantic label prediction to the observations many time instances away. However, the memory capacity of simple RNNs is limited because of the gradient vanishing and exploding problem. We propose to use an external memory to improve memorisation capability of RNNs. Experiments on the ATIS dataset demonstrated that the proposed model was able to achieve the state-of-the-art results. Detailed analysis may provide insights for future research.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wen Wang
2. Xiangyu Zeng
3. Hongxia Jin
4. Yilin Shen
5. Zhu Zhuo
6. Qian Chen
7. Yixuan Li
8. Zhenhua Ling

Citations in the corpus

• A Progressive Model to Enable Continual Learning for Semantic Slot Filling (2019, background): “They are designed to either train a slot filling model independently (Yao et al., 2014; Peng et al., 2015; Kurata et al., 2016; Hakkani-T¨ur et al., 2016; Liu and Lane, 2016; Deng et al., 2019; Ray et al., 2019) or jointly with the other intent detection task in SLU (Guo et al., 2014; Liu and Lane,…”
• Multi-Domain Adversarial Learning for Slot Filling in Spoken Language Understanding (2017, unknown): “Deep LSTM [16] 95.08 - - - - RNN-EM [17] 95.25 - - - - Encoder-labeler LSTM [18] 95.40 - - - 74.41 Attention Bi-LSTM [6] 95.75 - - - - BLSTM-LSTM (focus) [19] 95 2 shows the slot filling evaluation results.”
• BERT for Joint Intent Classification and Slot Filling (2019, background): “Approaches for slot filling include CNN (Vu, 2016), deep LSTM (Yao et al., 2014), RNN-EM (Peng et al., 2015), encoder-labeler deep LSTM (Kurata et al., 2016), and joint pointer and attention (Zhao and Feng, 2018), among others.”