RAND-WALK: A Latent Variable Model Approach to Word Embeddings

By Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, Andrej Risteski. 2015

67 citations recorded by the source corpus.

Subfields: Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Sanjeev Arora
2. Yuanzhi Li
3. Yingyu Liang
4. Andrej Risteski
5. Tengyu Ma
6. Suma Bhat
7. Pramod Viswanath
8. Jiaqi Mu

In Representation & Self-Supervised Learning

1. Yuanzhi Li
2. Sanjeev Arora
3. Tengyu Ma
4. Andrej Risteski
5. Yingyu Liang
6. Suma Bhat
7. Pramod Viswanath
8. Jiaqi Mu

Citations in the corpus

• Recovery guarantee of weighted low-rank approximation via alternating minimization (2016, background): “rix W is chosen according to prior knowledge about the noise. To list a few such applications, the co-occurrence matrix for words in natural language processing applications (Pennington et al., 2014; Arora et al., 2015) is such that the noise is larger when the co-occurrence of two words is rarer. I”
• Machine Learning Methods That Economists Should Know About (2019, background): “Athey [2017] and Athey et al. [2017c] provides a broader perspective with more emphasis on recent developments in adapting ML methods for causal questions and general implications for economics.”
• Cyclades: Conflict-free Asynchronous Machine Learning (2016, background): “A recent work by [2] proposes to solve: min{v”
• Linear Algebraic Structure of Word Senses, with Applications to Polysemy (2016, methodology): “f tie1, tie2, etc., but this fact alone seems insucient to extract the senses since v tiecan be expressed thus in innitely many ways. To make further progress we recall that the embedding method of Arora et al. (2015) is based upon a generative model of language that involves a random walk on discou”
• Representing Sentences as Low-Rank Subspaces (2017, background): “…word pairs tend to have similar difference vectors (Bengio et al., 2003; Mnih and Hinton, 2007; Mikolov et al., 2010; Collobert et al., 2011; Huang et al., 2012; Dhillon et al., 2012; Mikolov et al., 2013; Pennington et al., 2014; Levy and Goldberg, 2014; Arora et al., 2015; Stratos et al., 2015).”