Integrating Distributional Lexical Contrast into Word Embeddings for Antonym-Synonym Distinction

By Kim Anh Nguyen, Sabine Schulte im Walde, Ngoc Thang Vu. 2016

145 citations recorded by the source corpus.

We propose a novel vector representation that integrates lexical contrast into distributional vectors and strengthens the most salient features for determining degrees of word similarity. The improved vectors significantly outperform standard models and distinguish antonyms from synonyms with an average precision of 0.66-0.76 across word classes (adjectives, nouns, verbs). Moreover, we integrate the lexical contrast vectors into the objective function of a skip-gram model. The novel embedding outperforms state-of-the-art models on predicting word similarities in SimLex-999, and on distinguishing antonyms from synonyms.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

Memory-assisted prompt editing to improve GPT-3 after deployment (2022, methodology): “The synonyms (syn) and antonyms (ant) were obtained from Nguyen et al. (2016).”
Directional Skip-Gram: Explicitly Distinguishing Left and Right Context for Word Embeddings (2018, background): “To improve original word embedding models, there are various studies leveraging external knowledge to update word embeddings with post processing (Faruqui et al., 2015; Kiela et al., 2015; Song et al., 2017) or supervised objectives (Yu and Dredze, 2014; Nguyen et al., 2016).”
Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers (2020, background): “Joint pretraining models, on the one hand, augment distributional LM objectives with additional objectives based on external resources (Yu and Dredze, 2014; Nguyen et al., 2016; Lauscher et al., 2019) and train the extended model from scratch.”
Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints (2017, methodology): “…et al. (2015) modify the skip-gram objective function to ﬁne-tune word vectors by injecting paraphrasing constraints from PPDB. Mrkši´c et al. (2016) build on the retroﬁtting approach by jointly injecting synonymy and antonymy constraints; the same idea is reassessed by Nguyen et al. (2016).”
LogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs (2019, methodology): “We apply dLCE [Nguyen et al., 2016], which is a distributional lexical-contrast embedding model, to generate word vectors that distributedly represent the words in templates.”
Intent detection using semantically enriched word embeddings (2016, methodology): “However, these enrichment approaches showed improved performance only on word-level tasks such as word similarity [8, 9, 11], antonym detection [10, 11, 16, 13], semantic hierarchies [12, 15], and semantic scale inference [14].”
Embedding Words and Senses Together via Joint Knowledge-Enhanced Training (2016, background): “ing low similarity scores for antonym pairs (Pham et al., 2015; Schwartz et al., 2015; Nguyen et al., 2016; Mrksic et al., 2017), but this is outside the scope of this paper.”
Rotated Word Vector Representations and their Interpretability (2017, background): “, 2015; Woodsend and Lapata, 2015), with incorporating external information to them (Tian et al., 2016; Nguyen et al., 2016).”
LogBERT: Log Anomaly Detection via BERT (2021, methodology): “The template2vec uses a distributional lexical-contrast embedding model called dLCE [47] to generate vectors to represent words in log keys.”
SemEval-2017 Task 2: Multilingual and Cross-lingual Semantic Word Similarity (2017, background): “Indeed, several recent works have shown how signiﬁcant performance improvements can be obtained on this dataset by simply tweaking usual word embedding approaches to handle antonymy (Schwartz et al., 2015; Pham et al., 2015; Nguyen et al., 2016).”