MGNC-CNN: A Simple Approach to Exploiting Multiple Word Embeddings for Sentence Classification
By Ye Zhang, Stephen Roller, Byron C. Wallace. 2016
110 citations recorded by the source corpus.
We introduce a novel, simple convolution neural network (CNN) architecture - multi-group norm constraint CNN (MGNC-CNN) that capitalizes on multiple sets of word embeddings for sentence classification. MGNC-CNN extracts features from input embedding sets independently and then joins these at the penultimate layer in the network to form a final feature vector. We then adopt a group regularization strategy that differentially penalizes weights associated with the subcomponents generated from the respective embedding sets. This model is much simpler than comparable alternative architectures and requires substantially less training time. Furthermore, it is flexible in that it does not require input word embeddings to be of the same dimensionality. We show that MGNC-CNN consistently outperforms baseline models.
Subfields: Artificial Intelligence, Information Systems
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Iryna Gurevych
- Byron C. Wallace
- Matthew Lease
- Andreas Ruckl'e
- Steffen Eger
- Maxime Peyrard
- Ye Zhang
- Brandon Dang
In Information Systems
- Matthew Lease
- Byron C. Wallace
- Ye Zhang
- Andreas Ruckl'e
- Maxime Peyrard
- Steffen Eger
- Dan Xu
- Edward Banner
Citations in the corpus
- A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification (2015, background): “For how to better utilize multiple sets of embeddings, we refer to (Zhang et al., 2016).”
- Exploiting Domain Knowledge via Grouped Weight Sharing with Application to Text Categorization (2017, methodology): “To exploit both grouped and independent word weights, we adopt a two-channel CNN model (Zhang et al., 2016).”
- Concatenated Power Mean Word Embeddings as Universal Cross-Lingual Sentence Representations (2018, background): “This so-called direct transfer approach (Zhang et al., 2016) on sentence-level can be beneficial when labeled data is scarce, because sentencelevel approaches typically outperform task-specific sentence representations induced from word-level models in this case (Subramanian et al.”
- Neural information retrieval: at the end of the early years (2017, methodology): “Zheng and Callan (2015) propose direct modeling of bigrams and proximity terms.”