Recurrent neural networks for language understanding

By Kaisheng Yao, Geoffry Zweig, Mei-Yuh Hwang, Yangyang Shi, Dong Yu. 2013

328 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Model Architectures

Citations in the corpus

Efficient Natural Language Response Suggestion for Smart Reply (2017, methodology): “For example, neural network models have been used to learn more robust parsers [14, 24, 29].”
Neural Approaches to Conversational AI (2018, positive): “hows an ATIS (Airline Travel Information System) utterance example in the Inside-Outside-Beginning (IOB) format (Ramshaw and Marcus, 1995), where for each word the model is to predict a semantic tag. Yao et al. (2013) and Mesnil et al. (2015) applied recurrent neural networks to slot tagging, where ”
Deep Learning: Methods and Applications (2014, background): “[403] reported the success of RNNs in spoken language understanding.”
Sequence-to-sequence neural net models for grapheme-to-phoneme conversion (2015, background): “…a side-conditioned language model [11] is attractive for its simplicity, and apparent performance, and these successes complement other recent work in which neural networks have advanced the state-of-the-art, for example in language modeling [12, 13], language understanding [14], and parsing [15].”
A Survey on Dialogue Systems: Recent Advances and New Frontiers (2017, positive): “g of a sequence of words, and the output is a sequence of slot/concept IDs, one for each word. [17] and [15] used deep belief networks (DBNs), and achieved superior results compared to CRF baselines. [51; 115; 66; 113] applied RNN for slot ﬁlling. The semantic representation generated by NLU is furt”
Convolutional neural network based triangular CRF for joint intent detection and slot filling (2013, methodology): “To the best of our knowledge, this is the first attempt to train intent and slot models jointly based on NNs, and can be thought of as a continuous space version of the TriCRF model.”
When large language models meet personalization: perspectives of challenges and opportunities (2023, neutral): “For task-oriented dialogue systems, a common approach [185], [186] is to treat the response generation as a pipeline and handle it separately using four components: dialogue understanding [187], [188], dialogue state tracking [189], [190], [191], dialogue policy learning [192], [185], and natural la”
Toward Mention Detection Robustness with Recurrent Neural Networks (2016, methodology): “For NER, Collobert et al. (2011) propose a CNN-based framework while Mesnil et al. (2013) and Yao et al. (2013; 2014) investigate the RNNs for the slot ﬁlling problem in spoken language understanding.”
Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding (2013, methodology): “Future work will explore more efficient training of RNNs and the choice of more comprehensive features [28] and using a different RNN training toolkit [14] incorporating more advanced features.”
Advances and Challenges in Conversational Recommender Systems: A Survey (2021, background): “Answering Yes/No for an attributes Yes [100] Providing an utterance Yes [28, 119] No [266, 112] Providing preferred attribute values Yes [230] No [141]”
A Joint Model of Intent Determination and Slot Filling for Spoken Language Understanding (2016, methodology): “For ATIS, as in [Yao et al., 2013], to deal with unseen words in the test set, we marked all words with only one occurrence in the training data as hUNKi and used this label to represent those unseen words in the test set.”
Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling (2016, methodology): “…2009), Recurrent Neural Network (RNN) (Elman, 1990; Jordan, 1997) and one of its speciﬁc architectures, Long Short-Term Memory (LSTM) (Hochreiter and Schmidhuber, 1997), have been widely used since they can capture temporal dependencies (Yao et al., 2013; Yao et al., 2014a; Mesnil et al., 2015).”
Recurrent Neural Networks with External Memory for Spoken Language Understanding (2015, methodology): “Speciﬁcally, recurrent neural networks (RNNs) based methods have shown strong performances, in language modeling [16], language understanding [25], and machine translation [7,5] tasks.”
Translation Quality Estimation using Recurrent Neural Network (2016, methodology): “The architecture of RNN-LM has been used for Natural Language Understanding (NLU) (Yao et al., 2013; Yao et al., 2014) earlier.”
Sequence-to-Sequence Data Augmentation for Dialogue Language Understanding (2018, background): “Language understanding, as an important component in the task-oriented dialogue system pipeline, has drawn a lot of research attention in recent year, especially when enhanced by the rich representation power of the neural network, like recurrent neural network, LSTM (Yao et al., 2013; Yao et al., 2”
Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding (2015, methodology): “We used a context window of 3 for bag-of-word feature [24].”