Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling

By Gakuto Kurata, Bing Xiang, Bowen Zhou, Mo Yu. 2016

130 citations recorded by the source corpus.

Recurrent Neural Network (RNN) and one of its specific architectures, Long Short-Term Memory (LSTM), have been widely used for sequence labeling. In this paper, we first enhance LSTM-based sequence labeling to explicitly model label dependencies. Then we propose another enhancement to incorporate the global information spanning over the whole input sequence. The latter proposed method, encoder-labeler LSTM, first encodes the whole input sequence into a fixed length vector with the encoder LSTM, and then uses this encoded vector as the initial state of another LSTM for sequence labeling. Combining these methods, we can predict the label sequence with considering label dependencies and information of whole input sequence. In the experiments of a slot filling task, which is an essential component of natural language understanding, with using the standard ATIS corpus, we achieved the state-of-the-art F1-score of 95.66%.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Wanxiang Che
2. Hongxia Jin
3. Yilin Shen
4. Yutai Hou
5. Xiangyu Zeng
6. Yu Wang
7. Ting-Wei Liu
8. Piero Molino

Citations in the corpus

• A Progressive Model to Enable Continual Learning for Semantic Slot Filling (2019, background): “They are designed to either train a slot filling model independently (Yao et al., 2014; Peng et al., 2015; Kurata et al., 2016; Hakkani-Tür et al., 2016; Liu and Lane, 2016; Deng et al., 2019; Ray et al., 2019) or jointly with the other intent detection task in SLU (Guo et al., 2014; Liu and Lane,…”
• Collaborative Multi-Agent Dialogue Model Training Via Reinforcement Learning (2019, methodology): “…have been adopted for NLU, performing intent classi-fication and slot tagging both independently (T¨ur et al., 2012; Lee and Dernoncourt, 2016; Xu and Sarikaya, 2013; Mesnil et al., 2015; Kurata et al., 2016; Huang et al., 2015) and jointly (Zhang and Wang, 2016; Rojas-Barahona et al., 2016).”
• A Bi-Model Based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling (2018, methodology): “The most effective algorithms are based on the structures of sequence to sequence models (or ”encoder-decoder” models), and generate the intents and semantic tags either using separate models((Yao et al., 2014; Mesnil et al., 2015; Peng and Yao, 2015; Kurata et al., 2016; Hahn et al., 2011)) or a jo”
• Data Augmentation for Spoken Language Understanding via Joint Variational Generation (2018, methodology): “Particularly, performance gains from generated datasets have been studied and documented in the VQA task (Kafle, Yousefhussien, and Kanan 2017), general image classification (Ratner et al. 2017), and selected SLU tasks (Kurata, Xiang, and Zhou 2016; Hou et al. 2018).”
• Simple is Better! Lightweight Data Augmentation for Low Resource Slot Filling and Intent Classification (2020, methodology): “In order to allow comparison with more complex data augmentation approaches, we also report results obtained with state of the art approaches based on Seq2Seq (Hou et al., 2018a) and Conditional Variational Auto Encoder (CVAE) (Yoo et al., 2019).”
• Sequence-to-Sequence Data Augmentation for Dialogue Language Understanding (2018, methodology): “Kurata et al. (2016a) presented the only work we know that tried to augment data for LU.”