Dependency Based Embeddings for Sentence Classification Tasks
By Alexandros Komninos, Suresh Manandhar. 2016
153 citations recorded by the source corpus.
We compare different word embeddings from a standard window based skipgram model, a skipgram model trained using dependency context features and a novel skipgram variant that utilizes additional information from dependency graphs. We explore the effectiveness of the different types of word embeddings for word similarity and sentence classification tasks. We consider three common sentence classification tasks: question type classification on the TREC dataset, binary sentiment classification on Stanford’s Sentiment Treebank and semantic relation classification on the SemEval 2010 dataset. For each task we use three different classification methods: a Support Vector Machine, a Convolutional Neural Network and a Long Short Term Memory Network. Our experiments show that dependency based embeddings outperform standard window based embeddings in most of the settings, while using dependency context embeddings as additional features improves performance in all tasks regardless of the classification method. Ourembeddings and code are available at https:
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Iryna Gurevych
- Niklas Muennighoff
- Andreas Ruckl'e
- Nils Reimers
- Steffen Eger
- Maxime Peyrard
- Roland Vollgraf
- Alan Akbik
Citations in the corpus
- Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging (2017, methodology): “The embeddings by Komninos and Manandhar (2016) resulted in the best performance for the POS, the Entities and the Events task.”
- Contextual String Embeddings for Sequence Labeling (2018, methodology): “Following Reimers et al. (2017), we use GLOVE embeddings for English NER and KOMNIOS embeddings (Komninos and Manandhar, 2016) for PoS tagging and chunking.”
- Concatenated Power Mean Word Embeddings as Universal Cross-Lingual Sentence Representations (2018, background): “Extensions predict contexts given by dependency trees (Levy and Goldberg, 2014) or combinations of windows and dependency context (Komninos and Manandhar, 2016), leading to more syntactically oriented word embeddings.”
- MTEB: Massive Text Embedding Benchmark (2022, background): “(b) Non-transformer: Komninos (Komninos and Manandhar, 2016) and Glove (Pennington et al.”
- Choosing a Text Embedding Model: A Practical Benchmarking and Decision Framework (2026, unknown): “They weren’t designed specifically for retrieval or similarity — they’re general-purpose language representations (e.g., GloVe [21], Komninos embeddings [22], BERT [23]) that serve as a baseline for how much task-specific fine-tuning actually helps.”