Vector Space Models of Word Meaning and Phrase Meaning: A Survey

By K. Erk. 2012

340 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

A Neural Network for Factoid Question Answering over Paragraphs (2014, background): “Many computational models of compositionality focus on learning vector spaces (Zanzotto et al., 2010; Erk, 2012; Grefenstette et al., 2013; Yessenalina and Cardie, 2011).”
A Convolutional Neural Network for Modelling Sentences (2014, background): “In some cases, composition is defined by algebraic operations over word meaning vectors to produce sentence meaning vectors (Erk and Padó, 2008; Mitchell and Lapata, 2008; Mitchell and Lapata, 2010; Turney, 2012; Erk, 2012; Clarke, 2012).”
Compositionality Decomposed: How do Neural Networks Generalise? (2019, methodology): “The advancements of distributional semantics of the word level allowed the field of natural language processing to move from discrete mathematical methods to models that use continuous numerical vectors (see, e.g. Clark, 2015; Erk, 2012; Turney & Pantel, 2010).”
Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors (2014, methodology): “However, it is worth pointing out that the evaluated parameter subset encompasses settings (narrow context window, positive PMI, SVD reduction) that have been found to be most effective in the systematic explorations of the parameter space conducted by Bul-linaria and Levy (2007; 2012).”
A Roadmap Towards Machine Intelligence (2015, background): “Incidentally, notice that once the machine can read text, it can also exploit distributional learning from large amounts of text (Erk, 2012; Mikolov et al., 2013; Turney and Pantel, 2010) to induce word and phrase representations addressing some of the challenging natural language phenomena we just…”
Multimodal Distributional Semantics (2013, neutral): “General overviews of semantic space models are provided by Clark (2013), Erk (2012), Manning and Schütze (1999), Sahlgren (2006) and Turney and Pantel (2010).”
Distributional vectors encode referential attributes (2015, background): “Distributional models induce vector-based semantic representations of words from their contextual distributions in corpora, exploiting the observation that words with related meanings tend to occur in similar linguistic contexts (Turney and Pantel, 2010; Erk, 2012).”
Emerging Language Spaces Learned From Massively Multilingual Corpora (2018, background): “, [6,7]) and applied to a number of practical tasks.”
Building a shared world: mapping distributional to model-theoretic semantic spaces (2015, background): “While distributional semantics (Turney and Pantel, 2010; Clark, 2012; Erk, 2012) has proved very successful in modelling lexical effects such as graded similarity and polysemy, it clearly has difficulties accounting for logical phenomena which are well covered by”
Formal Distributional Semantics: Introduction to the Special Issue (2016, background): “Distributional Semantics (Turney and Pantel 2010; Clark 2012; Erk 2012) has a radically different view of language, based on the hypothesis that the meaning of a linguistic expression can be induced from the contexts in which it is used (Harris 1954; Firth 1957), because related expressions, such…”
Word Embeddings, Analogies, and Machine Learning: Beyond king - man + woman = queen (2016, background): “Word embeddings represent words in the vocabulary as vectors that can be derived directly from co-occurrence counts (“explicit models”) or learned implicitly by neural nets (see (Erk, 2012) for general overview of the ﬁeld).”
A Natural Proof System for Natural Language (2017, background): “vector space semantics (Turney and Pantel, 2010; Erk, 2012), can be easily integrated in the natural tableau system for modeling lexical semantics.”
High-risk learning: acquiring new word vectors from tiny data (2017, background): “Distributional models (DS: Turney and Pantel (2010); Clark (2012); Erk (2012)), and in particular neural network approaches (Bengio et al., 2003; Collobert et al., 2011; Huang et al., 2012; Mikolov et al., 2013), do not fare well in the absence of large corpora.”
Word meaning in minds and machines (2020, background): “Moreover, simpler models can also disambiguate some polysemous words through their surrounding words, either by computing the average embedding of the neighbors (Erk, 2012) or by inferring the latent topic (Griffiths et al., 2007).”
From Word to Sense Embeddings: A Survey on Vector Representations of Meaning (2018, background): “The optimal way of partitioning the meanings of words into multiple senses has long been the point of argument (Erk, McCarthy, & Gaylord, 2009; Erk, 2012; McCarthy, Apidianaki, & Erk, 2016).”
Distributional Models of Word Meaning (2018, methodology): “DSMs are typically evaluated for (a) accuracy in multiple-choice synonym detection tasks like the one used in the Test of English as a Foreign Language (TOEFL) (Landauer & Dumais 1997) and (b) correlation with human similarity ratings collected in data sets such as the small RG65, used in the semina”
“Look, some Green Circles!”: Learning to Quantify from Images (2016, background): “…of both a) a computational theory of meaning rooted in situated language use, as pursued by the ﬁeld of Distributional Semantics (Clark, 2012; Erk, 2012) and b) vision-based applications such as image caption generation and visual question answering (Antol et al., 2015), going towards…”
Compositional-ly Derived Representations of Morphologically Complex Words in Distributional Semantics (2013, background): “Recent surveys include Clark (2012), Erk (2012) and Turney and Pantel (2010).”
Book Reviews: Ontology-Based Interpretation of Natural Language by Philipp Cimiano, Christina Unger and John McCrae (2014, background): “In general, there has been a lot of work on representing the meaning of words using (distributional) vector space models [Erk, 2012].”