GloVe: Global Vectors for Word Representation

By Jeffrey Pennington, Richard Socher, Christopher D. Manning. 2014

34,790 citations recorded by the source corpus.

Recent methods for learning vector space representations of words have succeeded in capturing fine-grained semantic and syntactic regularities using vector arithmetic, but the origin of these regularities has remained opaque. We analyze and make explicit the model properties needed for such regularities to emerge in word vectors. The result is a new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods. Our model efficiently leverages statistical information by training only on the nonzero elements in a word-word cooccurrence matrix, rather than on the entire sparse matrix or on individual context windows in a large corpus. The model produces a vector space with meaningful substructure, as evidenced by its performance of 75% on a recent word analogy task. It also outperforms related models on similarity tasks and named entity recognition.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Percy Liang
  4. Luke S. Zettlemoyer
  5. Yoshua Bengio
  6. Yejin Choi
  7. Graham Neubig
  8. Quoc V. Le

In Natural Language Processing

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Luke S. Zettlemoyer
  4. Yejin Choi
  5. Graham Neubig
  6. Percy Liang
  7. Richard Socher
  8. Noah A. Smith

In Representation & Self-Supervised Learning

  1. Yoshua Bengio
  2. Sebastian Ruder
  3. Trevor Darrell
  4. Ruslan Salakhutdinov
  5. Dacheng Tao
  6. Andrea Vedaldi
  7. Jianfeng Gao
  8. Ivan Vulic

Citations in the corpus