Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding

By Grégoire Mesnil, Yann Dauphin, Kaisheng Yao, Yoshua Bengio, Li Deng, Dilek Z. Hakkani-Tür, Xiaodong He, Larry Heck, Gökhan Tür, Dong Yu and 1 more. 2015

587 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Information Extraction, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Li Deng
3. Bing Liu
4. Richard Socher
5. Minlie Huang
6. Yoshua Bengio
7. Xiaodong He
8. Mohit Bansal

In Natural Language Processing

1. Jianfeng Gao
2. Bing Liu
3. Richard Socher
4. Li Deng
5. Minlie Huang
6. Xiaodong He
7. Erik Cambria
8. Dilek Z. Hakkani-Tür

In Information Extraction

1. Bing Liu
2. Ian Lane
3. Hongxia Jin
4. Yilin Shen
5. Yu Wang
6. Xiangyu Zeng
7. Young-Kil Kim
8. Yoon-Hyung Roh

In Speech & Audio

1. Yoshua Bengio
2. Mirco Ravanelli
3. Li Deng
4. Ian Lane
5. Geoffry Zweig
6. Kaisheng Yao
7. Bing Liu
8. Hung-yi Lee

Citations in the corpus

• Efficient Natural Language Response Suggestion for Smart Reply (2017, neutral): “For example, neural network models have been used to learn more robust parsers [14, 24, 29].”
• Towards Open Intent Discovery for Conversational Text (2019, methodology): “Recent approaches have drawn upon progress in CNN, RNN, and CRF based language models to improve intent detection [11, 22, 24, 31, 36, 49, 52].”
• MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling (2018, methodology): “…of using statistical framework and machine learning for various system components, such as natural language understanding (Henderson et al., 2013; Mesnil et al., 2015; Mrkšić et al., 2017a), dialogue management (Gašić and Young, 2014; Tegho et al., 2018), language generation (Wen et al.,…”
• Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling (2016, methodology): “RNNs have been widely used in many sequence modeling problems [6, 13].”
• Attention with Intention for a Neural Network Conversation Model (2015, background): “They have made significant progresses in machine translation [1,6,13], language understanding [8], and speech recognition [4].”
• Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems (2018, background): “We believe that it is because the dialogue state space is too complex for the agent to learn from scratch, as it has to learn a good NLU model in combination with a good policy to complete the task Average Dialogue Turn Size Figure 4 shows the curves for the average turn size of successful dialogues”
• Neural Belief Tracker: Data-Driven Dialogue State Tracking (2016, neutral): “As shown by Mrkˇsi´c et al. (2015), such parameter tying can facilitate transfer learning and lead to improved belief tracking performance.”
• Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces (2018, background): “Recently, more computationally demanding approaches based on deep learning models have been proposed [31, 30].”
• From Eliza to XiaoIce: challenges and opportunities with social chatbots (2018, background): “…AI technologies, e.g., natural language understanding (Bengio et al., 2001; Mikolov et al., 2013; Sutskever et al., 2014; Bahdanau et al., 2015; Mesnil et al., 2015), speech recognition and synthesis (Hinton et al., 2012; Deng et al., 2013; Xiong et al., 2016; Qian et al., 2014; van den Oord…”
• Iterative policy learning in end-to-end trainable task-oriented neural dialog models (2017, background): “Conventional task-oriented dialog systems are designed with complex pipelines [4, 5, 6, 7], and there are usually separately developed modules for natural language understanding (NLU) [8, 9, 10, 11], dialog state tracking (DST) [12, 13], and dialog management (DM) [14, 15, 16].”
• Data Augmentation by Data Noising for Open-vocabulary Slots in Spoken Language Understanding (2019, methodology): “Noised context word window: Mesnil et al. (2015) and Zhang and Wang (2016) used a context word window to improve the performance of the RNNs in slot filling.”
• Speech Model Pre-training for End-to-End Spoken Language Understanding (2019, methodology): “In text-based NLU, this scenario can be handled using word embeddings, which represent words in such a way that words with similar meanings have similar vector representations [2, 43].”
• Data Augmentation for Spoken Language Understanding via Joint Variational Generation (2018, methodology): “Plain RNNs and LSTMs were first explored in (Mesnil et al. 2015; Yao et al. 2014).”
• Iterative Delexicalization for Improved Spoken Language Understanding (2019, methodology): “…parsing result for the user utterance x. Recurrent neural network parsers: RNN based parsers have been successfully applied for intent classification and slot labeling tasks in recent years [Mesnil et al., 2015; Hakkani-Tür et al., 2016; Liu and Lane, 2016; Kim et al., 2017a; Goo et al., 2018].”
• User Information Augmented Semantic Frame Parsing Using Progressive Neural Networks (2018, methodology): “Existing approaches includes the independent models for learning intent detection [1, 2] and slot filling [3, 4, 5, 6, 7, 8] separately as well as joint models to learn these two tasks together [9, 10, 11, 2].”
• Conversational Recommender System (2018, methodology): “Mesnil et al. [13] used LSTM and Conditional Random Fields networks to perform the slot filling.”