Revisiting Word Embedding for Contrasting Meaning

By Zhigang Chen, Wei Lin, Qian Chen, Xiaoping Chen, Si Wei, Hui Jiang, Xiao-Dan Zhu. 2015

47 citations recorded by the source corpus.

Contrasting meaning is a basic aspect of semantics. Recent word-embedding models based on distributional semantics hypothesis are known to be weak for modeling lexical contrast. We present in this paper the embedding models that achieve an F-score of 92% on the widely-used, publicly available dataset, the GRE “most contrasting word” questions (Mohammad et al., 2008). This is the highest performance seen so far on this dataset. Surprisingly at the first glance, unlike what was suggested in most previous work, where relatedness statistics learned from corpora is claimed to yield extra gains over lexicon-based models, we obtained our best result relying solely on lexical resources (Roget’s and WordNet)—corpora statistics did not lead to further improvement. However, this should not be simply taken as that distributional statistics is not useful. We examine several basic concerns in modeling contrasting meaning to provide detailed analysis, with the aim to shed some light on the future directions for this basic semantics modeling problem.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Mohit Bansal
2. Ting-Wei Liu
3. Xiao-Dan Zhu
4. Jianfeng Gao
5. Furu Wei
6. Zhenhua Ling
7. Bing Qin
8. Qian Chen

Citations in the corpus

• Leverage Financial News to Predict Stock Price Movements Using Word Embeddings and Deep Neural Networks (2015, methodology): “In this paper, we propose to use the recent word embedding methods (Mikolov et al., 2013b; Liu et al., 2015; Chen et al., 2015) to select features from on-line financial news corpora, and employ deep neural networks (DNNs) to predict the future stock movements based on the extracted features.”
• Distraction-Based Neural Networks for Modeling Document (2016, background): “[Chen et al., 2015] Zhigang Chen, Wei Lin, Qian Chen, Si Wei, Hui Jiang, and Xiaodan Zhu.”
• Sentiment Embeddings with Applications to Sentiment Analysis (2016, background): “Some recent studies [43], [51], [52], [53] also use neural networks to encode additional evidences to word embeddings.”
• Enhanced LSTM for Natural Language Inference (2016, background): “Future work interesting to us includes exploring the usefulness of external resources such as Word-Net and contrasting-meaning embedding (Chen et al., 2015) to help increase the coverage of word-level inference relations.”
• Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference (2017, background): “Future work on NLI includes exploring the usefulness of external resources such as Word-Net and contrasting-meaning embedding (Chen et al., 2015).”
• Neural Natural Language Inference Models Enhanced with External Knowledge (2017, background): “In general, external knowledge has shown to be effective in neural networks for other NLP tasks, including word embedding (Chen et al., 2015; Faruqui et al., 2015; Liu et al., 2015; Wieting et al., 2015; Mrksic et al., 2017), machine translation (Shi et al.”
• Learning Semantic Word Embeddings based on Ordinal Knowledge Constraints (2015, methodology): “…been widely used in various NLP tasks, including neural language model (Ben-gio et al., 2003; Schwenk, 2007), sequence labelling tasks (Collobert and Weston, 2008; Col-lobert et al., 2011), machine translation (Devlin et al., 2014; Sutskever et al., 2014), and antonym selection (Chen et al., 2015).”
• Intent detection using semantically enriched word embeddings (2016, methodology): “However, these enrichment approaches showed improved performance only on word-level tasks such as word similarity [8, 9, 11], antonym detection [10, 11, 16, 13], semantic hierarchies [12, 15], and semantic scale inference [14].”