A Structured Vector Space Model for Word Meaning in Context

By K. Erk, Sebastian Padó. 2008

405 citations recorded by the source corpus.

We address the task of computing vector space representations for the meaning of word occurrences, which can vary widely according to context. This task is a crucial step towards a robust, vector-based compositional account of sentence meaning. We argue that existing models for this task do not take syntactic structure sufficiently into account. We present a novel structured vector space model that addresses these issues by incorporating the selectional preferences for words' argument positions. This makes it possible to integrate syntax into the computation of word meaning in context. In addition, the model performs at and above the state of the art for modeling the contextual adequacy of paraphrases.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank (2013, background): “Variants of this idea use more complex frequencies such as how often a word appears in a certain syntactic context (Pado and Lapata, 2007; Erk and Padó, 2008).”
Experience Grounds Language (2020, background): “ture and composition (Erk and Padó, 2008; Socher et al., 2012) span decades of research.”
A Convolutional Neural Network for Modelling Sentences (2014, background): “In some cases, composition is defined by algebraic operations over word meaning vectors to produce sentence meaning vectors (Erk and Padó, 2008; Mitchell and Lapata, 2008; Mitchell and Lapata, 2010; Turney, 2012; Erk, 2012; Clarke, 2012).”
Zero-Shot Learning Through Cross-Modal Transfer (2013, background): “In these approaches, words are represented as vectors of distributional characteristics – most often their co-occurrences with words in context [26, 9, 1, 32].”
From Frequency to Meaning: Vector Space Models of Semantics (2010, background): “Deerwester et al. (1990) showed how the intuitions of Wittgenstein (1953), Harris (1954), Weaver, and Firth could be used in a practical algorithm.”
Deep Unordered Composition Rivals Syntactic Methods for Text Classification (2015, background): “Later, their work was extended to take into account the syntactic relation between words (Erk and Padó, 2008; Baroni and Zamparelli, 2010; Kartsaklis and Sadrzadeh, 2013) and grammars (Coecke et al.”
Multilingual Distributed Representations without Word Alignment (2013, background): “This was highlighted in [10], who proposed a mechanism for modifying a word’s representation based on its individual context.”
Distributional Memory: A General Framework for Corpus-Based Semantics (2010, neutral): “These include compositionality, but also modulating word meaning in context (Erk and Pad´o 2008; Mitchell and Lapata 2008) and ﬁnding ways to embed the distributional memory in complex NLP systems (e.g., for question answering or textual entailment) or even embodied agents and robots.”
Multilingual Models for Compositional Distributed Semantics (2014, background): “, 2003), paraphrase detection (Erk and Padó, 2008), and dialogue analysis (Kalchbrenner and Blunsom, 2013).”
Jointly optimizing word representations for lexical and sentential tasks with the C-PHRASE model (2015, background): “This makes C-PHRASE as simple as additive and multiplicative composition (Mitchell and Lapata, 2010), 3 but C-PHRASE is both more effective in compositional tasks (see evaluation below), and it has the further advantage that it learns its own word vectors, thus reducing the number of arbitrary…”
Multi-Step Regression Learning for Compositional Distributional Semantics (2013, background): “Several studies tackle word meaning in context, that is, how to adapt the distributional representation of a word to the speciﬁc context in which it appears (e.g., Dinu and Lapata, 2010; Erk and Pad´o, 2008; Thater et al., 2011).”
Domain and Function: A Dual-Space Model of Semantic Relations and Compositions (2012, background): “One application for semantic composition is measuring the similarity of phrases (Erk & Padó, 2008; Mitchell & Lapata, 2010).”
Compositional Operators in Distributional Semantics (2014, background): “Following a different path, Erk and Padó (2008) argue that a single vector is not enough to catch the meaning of a word; instead, the vector of a word is accompanied by a set of vectors R representing the lexical preferences of the word for its arguments positions, and a set of vectors R−1,…”
Analysing Lexical Semantic Change with Contextualised Word Representations (2020, background): “Padó (2008) proposed creating multiple vectors for the same word and Erk and Padó (2010) proposed to directly learn usage-specific representations based on the set of exemplary contexts within which the target word occurs.”
Distributional Models of Word Meaning (2018, background): “Erk & Padó (2008) address the fact that, when words are composed, they tend to affect one another’s meanings.”
Vector Space Models of Word Meaning and Phrase Meaning: A Survey (2012, background): “Another approach to the polysemy problem is to use clustering (Dinu and Lapata 2010; Erk and Padó 2010; Reisinger and Mooney 2010).”
Learning to Represent Words in Context with Multilingual Supervision (2015, background): “The second solution, which is the basis of this work, eschews sense inventories (whether latent or explicit) and says that lexical meaning is a function of word and its context (Erk & Padó, 2008; Kintsch, 2001; Mitchell & Lapta, 2008).”
Text: now in 2D! A framework for lexical expansion with contextual similarity (2013, background): “…a set of given alternatives by similarity of vector representation and context (e.g. word sense discrimination, Schütze 1998, synonyms, Rapp 2003, paraphrases , Erk and Padó 2008, word sense disambiguation, Thater et al. 2011), these tasks presuppose an existing list of alternatives to begin with.”
Detecting Asymmetric Semantic Relations in Context: A Case-Study on Hypernymy Detection (2017, methodology): “Following Thater et al. (2011); Erk and Pad´o (2008), we apply a ﬁlter to word type representations to highlight the salient dimensions of the exemplar context, emphasizing relevant dimensions and downplaying unimportant ones.”