Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding

By Grégoire Mesnil, Xiaodong He, Li Deng, Yoshua Bengio. 2013

444 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Speech & Audio

Citations in the corpus

Deep Learning: Methods and Applications (2014, background): “, 2013; Le and Mower, 2013), to spoken language understanding in (Mesnil et al., 2013; Yao et al., 2013; Tur et al., 2012), to speaker recognition in (Vasilakakis et al.”
Sequence-to-sequence neural net models for grapheme-to-phoneme conversion (2015, methodology): “The idea has been used for speech recognition [28] and more recently for language understanding [29].”
Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates (2017, background): “Recurrent neural networks (RNNs) which can handle the sequential data with variable lengths [13], [14], have shown their strength in language modeling [15], [16], [17], image captioning [18], [19], video analysis [20], [21], [22], [23], [24],”
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition (2016, background): “…neural networks (RNNs) which are a variant of neural nets for handling sequential data with variable length, have been successfully applied to language modeling [10–12], image captioning [13,14], video analysis [15–24], human re-identiﬁcation [25,26], and RGB-based action recognition [27–29].”
Bidirectional LSTM-CRF Models for Sequence Tagging (2015, background): “In speech language understanding community, recurrent neural network (Mesnil et al., 2013; Yao et al., 2014) and convolutional nets (Xu and Sarikaya, 2013) based models have been recently proposed.”
Crafting adversarial input sequences for recurrent neural networks (2016, methodology): “We convert these integer sequences to matrices, where each row encodes a word as a set of 128 coordinates— known as word embeddings [23], [24].”
Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval (2015, methodology): “The main idea of using RNN for sentence embedding is to ﬁnd a dense and low dimensional semantic representation by sequentially and recurrently processing each word in a sentence and mapping it into a low dimensional vector.”
A Survey on Dialogue Systems: Recent Advances and New Frontiers (2017, positive): “g of a sequence of words, and the output is a sequence of slot/concept IDs, one for each word. [17] and [15] used deep belief networks (DBNs), and achieved superior results compared to CRF baselines. [51; 115; 66; 113] applied RNN for slot ﬁlling. The semantic representation generated by NLU is furt”
A Model of Coherence Based on Distributed Sentence Representation (2014, methodology): “It has been successfully applied to tasks such as language modeling (Mikolov et al., 2010) and spoken language understanding (Mesnil et al., 2013).”
A tutorial survey of architectures, algorithms, and applications for deep learning (2014, background): “More recently, Mesnil et al. [52] reported the success of RNNs in spoken language understanding.”
Natural Language Processing Advancements By Deep Learning: A Survey (2020, methodology): “The application of deep neural networks in NER has been investigated by the employment of CNN [107] and RNN architectures [108], as well as hybrid bidirectional LSTM and CNN architectures [14].”
A Joint Model of Intent Determination and Slot Filling for Spoken Language Understanding (2016, methodology): “RNNs have shown excellent performance on the SF task [Mesnil et al., 2013; 2015] and outperform traditional models, such as CRF. Yao et al. [2014] improved RNNs by using transition features and the sequence-level optimization criterion of CRF to explicitly model dependencies of output labels.”
Towards more variation in text generation: Developing and evaluating variation models for choice of referential form (2016, result): “Our approach here is similar to the one presented by Mesnil et al. (2013).”