Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD

By John A. Bullinaria, Joseph P. Levy. 2012

329 citations recorded by the source corpus.

Subfields: Artificial Intelligence, General Social Sciences, Representation & Self-Supervised Learning, Computational Linguistics & Semantics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Social Sciences

In Representation & Self-Supervised Learning

In Computational Linguistics & Semantics

Citations in the corpus

A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications (2017, background): “graph embedding (random walk involved) in certain tasks as the latter relies on separate local context windows [147], [148].”
Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors (2014, background): “…of extensive evaluations of alternative count DSMs on standard benchmarks (Agirre et al., 2009; Ba-roni and Lenci, 2010; Bullinaria and Levy, 2007; Bullinaria and Levy, 2012; Sahlgren, 2006; Pad´o and Lapata, 2007), the existing literature contains very little in terms of direct comparison of…”
Multimodal Distributional Semantics (2013, background): “…as: It is worth observing that, in an extensive study of how parameters affect the quality of semantic vectors, Bullinaria and Levy (2007) and Bullinaria and Levy (2012) found that a model similar to our Window2 (co-occurrence statistics from ukWaC, narrow window, lemmatized content word…”
Multi-Step Regression Learning for Compositional Distributional Semantics (2013, background): “Extensive evidence suggests that dimensionality reduction does not affect, and might even improve the quality of lexical semantic vectors (Bullinaria and Levy, 2012; Landauer and Dumais, 1997; Sahlgren, 2006; Sch¨utze, 1997).”
Deep Neural Networks for Learning Graph Representations (2016, methodology): “Such approaches learn representations based on statistics of global co-occurrence counts, and can outperform neural network method based on separate local context windows in certain prediction tasks (Bullinaria and Levy 2012; Levy, Goldberg, and Dagan 2015).”
Unsupervised Post-processing of Word Vectors via Conceptor Negation (2018, methodology): “5 are recommended in (Caron 2001; Bullinaria and Levy 2012; Levy, Goldberg, and Dagan 2015).”
Inducing Domain-Specific Sentiment Lexicons from Unlabeled Corpora (2016, methodology): “Excluding the singular value weights, Σ, has been shown known to dramatically improve embedding quality (Turney and Pantel, 2010; Bullinaria and Levy, 2012).”
Analogy-based detection of morphological and semantic relations with word embeddings: what works and what doesn’t. (2016, background): “So far they have been studied in the context of semantic priming (Lapesa and Evert, 2014), semantic similarity tasks (Kiela and Clark, 2014), and across groups of tasks (Bullinaria and Levy, 2012).”
Distributional Semantics Beyond Words: Supervised Learning of Analogy and Paraphrase (2013, methodology): “Optimal performance requires tuning the parameters k and p for the task (Bullinaria and Levy, 2012; Turney, 2012).”
SensEmbed: Learning Sense Embeddings for Word and Relational Similarity (2015, methodology): “The conventional way to obtain such representations is to compute a term-document occurrence matrix on large corpora and then reduce the dimensionality of the matrix using techniques such as singular value decomposition (Deerwester et al., 1990; Bullinaria and Levy, 2012, SVD).”
Uncovering Divergent Linguistic Information in Word Embeddings with Lessons for Intrinsic and Extrinsic Evaluation (2018, methodology): “The proposed method is also connected to a similar parameter found in traditional count-based distributional models as introduced by Caron (2001) and further analyzed by Bullinaria and Levy (2012) and Turney (2012).”
GraRep: Learning Graph Representations with Global Structural Information (2015, methodology): “The resulting W k gives representations of current vertices as its column vectors, and C gives the representations of context vertices as its column vectors [6, 5, 30].”
Distributional Models of Word Meaning (2018, methodology): “Performance greatly varies depending on the data set and the model, but DSMs can achieve perfect accuracy on the TOEFL (Bullinaria & Levy 2012) and a Spearman correlation of 0.8 or better with similarity ratings.”