Efficient Estimation of Word Representations in Vector Space

By Tomas Mikolov, Kai Chen, Greg S. Corrado, Joseph Dean. 2013

34,723 citations recorded by the source corpus.

We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previously best performing techniques based on different types of neural networks. We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high quality word vectors from a 1.6 billion words data set. Furthermore, we show that these vectors provide state-of-the-art performance on our test set for measuring syntactic and semantic word similarities.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Yoshua Bengio
  4. Percy Liang
  5. Luke S. Zettlemoyer
  6. Quoc V. Le
  7. Graham Neubig
  8. Kyunghyun Cho

In Natural Language Processing

  1. Mohit Bansal
  2. Graham Neubig
  3. Jianfeng Gao
  4. Luke S. Zettlemoyer
  5. Yejin Choi
  6. Percy Liang
  7. Kyunghyun Cho
  8. Noah A. Smith

In Representation & Self-Supervised Learning

  1. Yoshua Bengio
  2. Trevor Darrell
  3. Ruslan Salakhutdinov
  4. Sebastian Ruder
  5. James R. Glass
  6. Andrew Zisserman
  7. Dacheng Tao
  8. Quoc V. Le

Citations in the corpus