An Introduction to Random Indexing

By Magnus Sahlgren. 2005

512 citations recorded by the source corpus.

Word space models enjoy considerable attention in current research on semantic indexing. Most notably, Latent Semantic Analysis/Indexing (LSA/LSI; Deerwester et al., 1990, Landauer & Dumais, 1997) has become a household name in information access research, and deservedly so; LSA has proven its mettle in numerous applications, and has more or less spawned an entire research field since its introduction around 1990. Today, there is a rich flora of word space models available, and there are numerous publications that report exceptional results in many different applications, including information retrieval (Dumais et al., 1988), word sense disambiguation (Schutze, 1993), various semantic knowledge tests (Lund et al., 1995, Karlgren & Sahlgren, 2001), and text categorization (Sahlgren & Karlgren, 2004). This paper introduces the Random Indexing word space approach, which presents an efficient, scalable and incremental alternative to standard word space methods. The paper is organized as follows: in the next section, we review the basic word space methodology. We then look at some of the problems that are inherent in the basic methodology, and also review some of the solutions that have been proposed in the literature. In the final section, we introduce the Random Indexing word space approach, and briefly review some of the experimental results that have been achieved with Random Indexing.

View Paper: https://www.semanticscholar.org/paper/3ef1b0469b43acc8ede2e56d8f001ad090b04826

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Distributional Memory: A General Framework for Corpus-Based Semantics (2010, unknown): “The nearest neighbors are efﬁciently searched in the W 1 × LW 2 matrix by compressing it to 5,000 dimensions via random indexing, using the parameters suggested by Sahlgren (2005).”
Word Representations: A Simple and General Method for Semi-Supervised Learning (2010, methodology): “This random indexing method is motivated by the Johnson-Lindenstrauss lemma, which states that for certain choices of random matrix, if d is su ﬃ ciently large, then the original distances be-tween words in F will be preserved in f (Sahlgren, 2005).”
TOPSIG: topology preserving document signatures (2011, background): “Random Indexing (RI) [17] is an efficient, scalable and incremental approach to dimensionality reduction.”
Target-Centric Features for Translation Quality Estimation (2014, methodology): “The novel features we add to our systems are (a) a language model on a combination of stop words and POS tags, (b) inverse glass-box features for translating the translation, and (c) random indexing (Sahlgren, 2005) for measuring the semantic similarity of source and target side across languages.”
Semantic Vector Products: Some Initial Investigations (2008, methodology): “It is also interesting that this method for constructing semantic vectors, known sometimes as " Random Indexing " or " Random Projection " is also the most computationally tractable and thus likely to lead to commercial-scale applications (Sahlgren 2005; Widdows & Ferraro 2008).”
From Frequency to Meaning: Vector Space Models of Semantics (2010, background): “The idea of random projection is to take high-dimensional vectors and randomly project them into a relatively low-dimensional space (Sahlgren, 2005).”
A Survey of Automatic Query Expansion in Information Retrieval (2012, methodology): “1987], spreading activation networks [Crestani 1997], concept lattice-based IR [Carpineto and Romano 2004], random indexing [Sahlgren 2005], and contextual document ranking modeled as basis vectors [Melucci 2008].”
Multimodal Distributional Semantics (2013, background): “…of the matrix containing the semantic vectors as rows can optionally be decreased by dimensionality reduction, that might provide beneficial smoothing by getting rid of noise components and/or allow more efficient storage and computation (Landauer & Dumais, 1997; Sahlgren, 2005; Schütze, 1997).”