Producing high-dimensional semantic spaces from lexical co-occurrence

By K. Lund, C. Burgess. 1996

Subfields: Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

GloVe: Global Vectors for Word Representation (2014, methodology): “A main problem with HAL and related meth-ods is that the most frequent words contribute a disproportionate amount to the similarity measure: the number of times two words co-occur with the or and , for example, will have a large effect on their similarity despite conveying relatively little about th”
From Frequency to Meaning: Vector Space Models of Semantics (2010, methodology): “…Semantic Analysis (LSA) (Deerwester et al., 1990; Landauer & Dumais, 1997), Hyperspace Analogue to Language (HAL) (Lund, Burgess, & Atchley, 1995; Lund & Burgess, 1996), and related research (Landauer, McNamara, Dennis, & Kintsch, 2007) is entirely within the scope of VSMs, as defined above,…”
A semantic matching energy function for learning with multi-relational data (2013, methodology): “The application of SME to WSD is related to work on vector-based models of word meaning (Lund and Burgess 1996; Landauer and Dumais 1997) and neural language models (Bengio 2008; Collobert et al.”
Multimodal Distributional Semantics (2013, methodology): “We define context in terms of words that co-occur within a window of fixed width, in the tradition of the popular HAL model (Lund & Burgess, 1996).”
A Structured Vector Space Model for Word Meaning in Context (2008, background): “In a default semantic space as described above, each vector represents one lemma , averaging over all its possible usages (Landauer and Dumais, 1997; Lund and Burgess, 1996).”
Distributional Memory: A General Framework for Corpus-Based Semantics (2010, neutral): “…rising wave of interest among computational linguists and cognitive scientists in corpus-based models of semantic representation (Grefenstette 1994; Lund and Burgess 1996; Landauer and Dumais 1997; Sch¨utze 1997; Sahlgren 2006; Bullinaria and Levy 2007; Grifﬁths, Steyvers, and Tenenbaum 2007;…”
Deep Neural Networks for Learning Graph Representations (2016, methodology): “An example of a matrix factorization method is hyperspace analogue analysis (Lund and Burgess 1996) which factorizes a word-word co-occurrence matrix to yield word representations.”
Thinking in Words: Language as an Embodied Medium of Thought (2014, background): “…that extract statistical regularities from large corpuses, including the Latent Semantic Analysis model (Landauer & Dumais, 1997; Landauer, Foltz, & Laham, 1998), the Hyperspace Analog to Language model (Lund & Burgess, 1996), and the Latent Dirichlet Allocation model (Blei, Ng, & Jordan, 2003).”
The English Lexicon Project (2007, methodology): “Freq_HAL refers to the Hyperspace Analogue to Language (HAL) frequency norms (Lund & Burgess, 1996), based on the HAL corpus , which consists of approximately 131 million words gathered across 3,000 Usenet newsgroups during February 1995.”
Evolution of Semantic Similarity—A Survey (2020, background): “Hyperspace Analogue to Language(HAL) [54]: HAL builds word co-occurrence matrix that has both rows and columns representing the words in the vocabulary and the matrix elements are populated with association strength values.”
On the computational basis of learning and cognition: Arguments from LSA (2002, methodology): “…the case of words, this would take the form of computations based on words-to-following-words association matrices, as is done in the HAL model of Lund and Burgess (1996), the SP model of Simon Dennis (2001), and the language models used in automatic speech recognition (e.g. Rabiner, 1969;…”
Vector Space Models of Lexical Meaning (2015, background): “Chapter 18 of Manning et al. (2008) provides an excellent textbook treatment of matrix decompositions in the context of Information Retrieval.”
Distributional Semantics in Technicolor (2012, background): “The Window2 and Window20 models are based on counting co-occurrences with collocates within a window of ﬁxed width, in the tradition of HAL (Lund and Burgess, 1996).”
Word Representations: A Simple and General Method for Semi-Supervised Learning (2010, background): “Hyperspace Analogue to Language (HAL) is another early distributional approach (Lund et al., 1995; Lund & Burgess, 1996) to inducing word representations.”
Portuguese Word Embeddings: Evaluating on Word Analogies and Natural Language Tasks (2017, methodology): “They can be roughly divided into two families of methods [Baroni et al. 2014]: the ﬁrst is composed of methods that work with a co-occurrence word matrix, such as Latent Semantic Analysis (LSA) [Dumais et al. 1988], Hyperspace Analogue to Language (HAL) [Lund and Burgess 1996] and Global Vectors (Gl”