Learning Tag Embeddings and Tag-specific Composition Functions in Recursive Neural Network

By Qiao Qian, Bo Tian, Minlie Huang, Yang Liu, Xuan Zhu, Xiaoyan Zhu. 2015

70 citations recorded by the source corpus.

Recursive neural network is one of the most successful deep learning models for natural language processing due to the compositional nature of text. The model recursively composes the vector of a parent phrase from those of child words or phrases, with a key component named composition function. Although a variety of composition functions have been proposed, the syntactic information has not been fully encoded in the composition process. We propose two models, Tag Guided RNN (TGRNN for short) which chooses a composition function according to the part-ofspeech tag of a phrase, and Tag Embedded RNN/RNTN (TE-RNN/RNTN for short) which learns tag embeddings and then combines tag and word embeddings together. In the fine-grained sentiment classification, experiment results show the proposed models obtain remarkable improvement: TG-RNN/TE-RNN obtain remarkable improvement over baselines, TE-RNTN obtains the second best result among all the top performing models, and all the proposed models have much less parameters/complexity than their counterparts.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Shuai Wang
2. B. Liu
3. Bing Liu
4. Minlie Huang
5. Lei Zhang
6. Xiaoyan Zhu
7. Erik Cambria
8. Yoav Goldberg

Citations in the corpus

• Attention-based LSTM for Aspect-level Sentiment Classification (2016, methodology): “Classical models including Recursive Neural Network (Socher et al., 2011; Dong et al., 2014; Qian et al., 2015), Recursive Neural Tensor Network (Socher et al.”
• Neural Network Methods for Natural Language Processing (2017, methodology): “Such approach is taken by Qian et al. [2015]. An alternative approach, due to Socher et al. [2013a] is to untie the weights according to the non-terminals, using a different composition matrix for each B; C pair of symbols:3”
• SentiLARE: Linguistic Knowledge Enhanced Language Representation for Sentiment Analysis (2019, background): “For instance, existing work has shown that linguistic knowledge including part-of-speech tag (Qian et al., 2015; Huang et al., 2017) and word-level sentiment polarity (Qian et al., 2017) is closely related to the sentiment of longer texts.”
• Deep learning for sentiment analysis: A survey (2018, methodology): “An example process of vector composition in RecNN is shown in Figure 9 (Qian et al., 2015).”