Distributed Representations of Sentences and Documents

By Quoc V. Le, Tomas Mikolov. 2014

9,778 citations recorded by the source corpus.

Many machine learning algorithms require the input to be represented as a fixed-length feature vector. When it comes to texts, one of the most common fixed-length features is bag-of-words. Despite their popularity, bag-of-words features have two major weaknesses: they lose the ordering of the words and they also ignore semantics of the words. For example, "powerful," "strong" and "Paris" are equally distant. In this paper, we propose Paragraph Vector, an unsupervised algorithm that learns fixed-length feature representations from variable-length pieces of texts, such as sentences, paragraphs, and documents. Our algorithm represents each document by a dense vector which is trained to predict words in the document. Its construction gives our algorithm the potential to overcome the weaknesses of bag-of-words models. Empirical results show that Paragraph Vectors outperforms bag-of-words models as well as other techniques for text representations. Finally, we achieve new state-of-the-art results on several text classification and sentiment analysis tasks.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Percy Liang
  4. Luke S. Zettlemoyer
  5. Yoshua Bengio
  6. Graham Neubig
  7. Kyunghyun Cho
  8. Quoc V. Le

In Natural Language Processing

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Graham Neubig
  4. Luke S. Zettlemoyer
  5. Kyunghyun Cho
  6. Yejin Choi
  7. Percy Liang
  8. Noah A. Smith

In Representation & Self-Supervised Learning

  1. Yoshua Bengio
  2. Sebastian Ruder
  3. Ruslan Salakhutdinov
  4. Quoc V. Le
  5. Jianfeng Gao
  6. James R. Glass
  7. Honglak Lee
  8. Kyunghyun Cho

Citations in the corpus