Two/Too Simple Adaptations of Word2Vec for Syntax Problems

By Wang Ling, Chris Dyer, Alan W. Black, Isabel Trancoso. 2015

404 citations recorded by the source corpus.

We present two simple modifications to the models in the popular Word2Vec tool, in order to generate embeddings more suited to tasks involving syntax. The main issue with the original models is the fact that they are insensitive to word order. While order independence is useful for inducing semantic representations, this leads to suboptimal results when they are used to solve syntax-based problems. We show improvements in part-ofspeech tagging and dependency parsing using our proposed models.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Computational Linguistics & Semantics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Noah A. Smith
  2. Mohit Bansal
  3. Graham Neubig
  4. Yoav Goldberg
  5. Luke S. Zettlemoyer
  6. Percy Liang
  7. Yoshua Bengio
  8. Kyunghyun Cho

In Representation & Self-Supervised Learning

  1. Yoav Goldberg
  2. Chris Dyer
  3. Noah A. Smith
  4. Eduard H. Hovy
  5. Anders Søgaard
  6. Diego Marcheggiani
  7. Graham Neubig
  8. Yoshua Bengio

In Computational Linguistics & Semantics

  1. Yoav Goldberg
  2. Ivan Titov
  3. Diego Marcheggiani
  4. Carlos Gómez-Rodríguez
  5. Daniel Klein
  6. Daniel Fried
  7. Eduard H. Hovy
  8. Xuezhe Ma

Citations in the corpus