Extracting semantic representations from word co-occurrence statistics: A computational study

By John A. Bullinaria, Joseph P. Levy. 2007

788 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Developmental and Educational Psychology, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Developmental and Educational Psychology

In Natural Language Processing

Citations in the corpus

GloVe: Global Vectors for Word Representation (2014, background): “A variety of newer models also pursue this approach, including a study (Bullinaria and Levy, 2007) that indicates that positive pointwise mutual information (PPMI) is a good transformation.”
Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change (2016, methodology): “…avenue is the use of distributional semantics, in which words are embedded in vector spaces according to their co-occurrence relationships (Bullinaria and Levy, 2007; Turney and Pantel, 2010), and the embeddings of words Figure 1: Two-dimensional visualization of semantic change in…”
Semantics derived automatically from language corpora contain human-like biases (2016, neutral): “The general idea that text corpora capture semantics including cultural stereotypes and empirical associations has long been known in corpus linguistics (1,2), but our findings add to this knowledge in three ways.”
From Frequency to Meaning: Vector Space Models of Semantics (2010, background): “Bullinaria and Levy (2007) compared these five distance measures and the cosine similarity measure on four different tasks involving word similarity.”
Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors (2014, background): “…despite the long tradition of extensive evaluations of alternative count DSMs on standard benchmarks (Agirre et al., 2009; Baroni and Lenci, 2010; Bullinaria and Levy, 2007; Bullinaria and Levy, 2012; Sahlgren, 2006; Padó and Lapata, 2007), the existing literature contains very little in terms…”
Multimodal Distributional Semantics (2013, methodology): “Here we focus on cosine similarity since it has been shown to be a very effective measure on many semantic benchmarks (Bullinaria & Levy, 2007; Padó & Lapata, 2007).”
Linguistic Regularities in Sparse and Explicit Word Representations (2014, background): “The PPMI variant dates back to at least (Niwa and Nitta, 1994), and was demonstrated to perform very well in Bullinaria and Levy (2007).”
Distributional Memory: A General Framework for Corpus-Based Semantics (2010, methodology): “Moreover, DSMs for attributional and relational similarity are widely used for the semi-automatic bootstrapping or extension of terminological repositories, computational lexicons (e.g., WordNet), and ontologies (Buitelaar, Cimiano, and Magnini 2005; Lenci 2010).”
Deep Neural Networks for Learning Graph Representations (2016, methodology): “Next we calculate the PPMI matrix based on our probabilistic co-occurrence matrix by following (Bullinaria and Levy 2007) (See Sec 2.”
Domain and Function: A Dual-Space Model of Semantic Relations and Compositions (2012, background): “This is a variation of pointwise mutual information (PMI) (Church & Hanks, 1989; Turney, 2001) in which all PMI values that are less than zero are replaced with zero (Niwa & Nitta, 1994; Bullinaria & Levy, 2007).”
deepDR: a network-based deep learning approach to in silico drug repositioning (2019, methodology): “Next, we calculated a shifted positive pointwise mutual information (PPMI) matrix (Bullinaria and Levy, 2007).”
Diachronic word embeddings and semantic shifts: a survey (2018, background): “3 Hamilton et al. (2016c) showed the superiority of SGNS over explicit PPMI-based distributional models in semantic shifts analysis, although they noted that low-rank SVD approximations (Bullinaria and Levy, 2007) can perform on par with SGNS, especially on smaller datasets.”
Is this a wampimuk? Cross-modal mapping between distributional semantics and the visual world (2014, methodology): “The cosine has been widely used in the distributional semantic literature, and it has been shown to outperform Euclidean distance (Bullinaria and Levy, 2007).7 Parameters were estimated with standard backpropagation and L-BFGS.”
A distributional similarity approach to the detection of semantic change in the Google Books Ngram corpus. (2011, result): “The window of length 2 was chosen for practical reasons given the huge size or the Google Ngram corpus, but it has been shown to produce good results in previous studies (e.g. Bullinaria and Levy, 2007).”
GraRep: Learning Graph Representations with Global Structural Information (2015, background): “Levy et al. [4] presented matrix factorization over shifted positive Point-wise Mutual Information (PMI) matrix for learning word representations and showed that the Skip-Gram model with Negative Sampling (SGNS) can be regarded as a model that implicitly such a matrix [16].”
An Introduction to Neural Information Retrieval (2018, neutral): “, 2006], and [Bullinaria and Levy, 2007]), and neighbouring words with distance (e.”