Parsing Natural Scenes and Natural Language with Recursive Neural Networks

By Richard Socher, Cliff Chiung-Yu Lin, Andrew Y. Ng, Christopher D. Manning. 2011

1,490 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Grammar as a Foreign Language (2014, methodology): “[25] used a tree-structured neural network to score candidate parse trees.”
Distributed Representations of Words and Phrases and their Compositionality (2013, methodology): “Other techniques that aim to represent meaning of sentences by composing the w ord vectors, such as the recursive autoencoders [15], would also benefit from using phrase vect ors instead of the word vectors.”
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank (2013, methodology): “1 RNN: Recursive Neural Network The simplest member of this family of neural network models is the standard recursive neural network (Goller and Küchler, 1996; Socher et al., 2011a).”
Deep Fragment Embeddings for Bidirectional Image Sentence Mapping (2014, methodology): “In language domain, several neural network models have been proposed to learn word/n-gram representations [29, 30, 31, 32, 33, 34], sentence representations [35] and paragraph/document representations [36].”
Grounded Compositional Semantics for Finding and Describing Images with Sentences (2014, methodology): “RNN models (Pollack, 1990; Goller and K¨uchler, 1996; Socher et al., 2011b; Socher et al., 2011a) provided a novel way of combining word vectors for longer phrases that moved beyond simple averaging.”
Gated Graph Sequence Neural Networks (2015, methodology): “Similarly, Recursive Neural Networks (Goller & Kuchler, 1996; Socher et al., 2011) being extended to Tree LSTMs (Tai et al., 2015) is analogous to our using of GRU updates in GG-NNs instead of the standard GNN recurrence with the aim of improving the long-term propagation of information across a…”
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks (2015, methodology): “Lastly, tree-structured models compose each phrase and sentence representation from its constituent subphrases according to a given syntactic structure over the sentence (Goller and Kuchler, 1996; Socher et al., 2011).”
Enhanced LSTM for Natural Language Inference (2016, methodology): “, 2015; Le and Zuidema, 2015), which extends the chain LSTM to a recursive network (Socher et al., 2011).”
A Joint Sequence Fusion Model for Video Question Answering and Retrieval (2018, methodology): “[22,23] using recursive neural networks), but they require groundtruth parse trees or segmentation labels.”
Exploiting Similarities among Languages for Machine Translation (2013, background): “…and to various other NLP tasks such as word representation learning, named entity recognition, disambiguation, parsing, and tag-w ging (Collobert and Weston, 2008; Turian et al., 2010; Socher et al., 2011; Socher et al., 2013; Col-lobert et al., 2011; Huang et al., 2012; Mikolov et al., 2013a).”
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation (2015, background): “Now there is an active interest for semantic pixel-wise labelling [2], [3], [4],[6], [7], [8], [9], [10], [11], [12], [13], [14], [15].”
A Fast Unified Model for Parsing and Sentence Understanding (2016, background): “…al., 2014; Zhang et al., 2015), which accumulate information using ﬁl-ters over short local sequences of words or characters; and tree-structured recursive neural networks (TreeRNNs, Goller and K¨uchler, 1996; Socher et al., 2011a, see Figure 1b), which propagate information up a binary parse tree.”
Feedforward semantic segmentation with zoom-out features (2014, neutral): “Recent work closest to ours includes [10, 30, 34, 28, 16].”
Natural Language Inference over Interaction Space (2017, background): “In a TreeRNN setting, the interaction layer models the interaction between each node pair(Socher et al., 2011a).”
A fast and simple algorithm for training neural probabilistic language models (2012, methodology): “Word representations learned by language models are also used for natu-ral language processing applications such as semantic role labelling (Collobert & Weston, 2008), sentiment analysis (Maas & Ng, 2010), named entity recognition (Turian et al., 2010), and parsing (Socher et al., 2011).”
Distilling Task-Specific Knowledge from BERT into Simple Neural Networks (2019, background): “In the past, researchers have developed and applied various neural architectures for NLP, including convolutional neural networks (Kalchbrenner et al., 2014; Kim, 2014), recurrent neural networks (Mikolov et al., 2010, 2011; Graves, 2013), and recursive neural networks (Socher et al., 2010, 2011).”
Learning Deep Structured Models (2014, background): “Similarly state-of-the-art results have been demonstrated for natural language processing, e.g., in [26, 5, 6].”
Convolutional Neural Network Architectures for Matching Natural Language Sentences (2014, background): “However, unlike recursive models [20, 21], the convolutional architecture has a ﬁxed depth, which bounds the level of composition it could do.”
Improved Recurrent Neural Networks for Session-based Recommendations (2016, positive): “Deep learning has recently been applied very successfully in areas such as image recognition [16, 9], speech recognition [8, 1] and natural language processing [24].”