Enhancing the LexVec Distributed Word Representation Model Using Positional Contexts and External Memory

By Alexandre Salle, Marco Idiart, Aline Villavicencio. 2016

30 citations recorded by the source corpus.

In this paper we take a state-of-the-art model for distributed word representation that explicitly factorizes the positive pointwise mutual information (PPMI) matrix using window sampling and negative sampling and address two of its shortcomings. We improve syntactic performance by using positional contexts, and solve the need to store the PPMI matrix in memory by working on aggregate data in external memory. The effectiveness of both modifications is shown using word similarity and analogy tasks.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yejin Choi
2. Noah A. Smith
3. Maarten Sap
4. Mohit Bansal
5. Roy Schwartz
6. Luke S. Zettlemoyer
7. Hannaneh Hajishirzi
8. Yulia Tsvetkov

In Information Systems

1. Yejin Choi
2. Noah A. Smith
3. Leila Zilles
4. Roy Schwartz
5. Ioannis Konstas
6. Maarten Sap

In Natural Language Processing

1. Yejin Choi
2. Noah A. Smith
3. Maarten Sap
4. Mohit Bansal
5. Roy Schwartz
6. Yulia Tsvetkov
7. Luke S. Zettlemoyer
8. Ronan Le Bras

In Representation & Self-Supervised Learning

1. Noah A. Smith
2. Yejin Choi
3. Roy Schwartz
4. Ioannis Konstas
5. Maarten Sap

Citations in the corpus

• The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task (2017, background): “This finding provides strong evidence that the authors’ style was affected by the writing task they were given to perform.”
• Story Cloze Task: UW NLP System (2017, background): “3% better than the published state of the art (Salle et al., 2016).”