Graph Convolutional Encoders for Syntax-aware Neural Machine Translation

By Jasmijn Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, Khalil Sima'an. 2017

519 citations recorded by the source corpus.

We present a simple and effective approach to incorporating syntactic structure into neural attention-based encoder-decoder models for machine translation. We rely on graph-convolutional networks (GCNs), a recent class of neural networks developed for modeling graph-structured data. Our GCNs use predicted syntactic dependency trees of source sentences to produce representations of words (i.e. hidden states of the encoder) that are sensitive to their syntactic neighborhoods. GCNs take word representations as input and produce word representations as output, so they can easily be incorporated as layers into standard encoders (e.g., on top of bidirectional RNNs or convolutional neural networks). We evaluate their effectiveness with English-German and English-Czech translation experiments for different types of encoders and observe substantial improvements over their syntax-agnostic versions in all the considered setups.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Graph Learning, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Graham Neubig
  3. Jianfeng Gao
  4. Luke S. Zettlemoyer
  5. Percy Liang
  6. Noah A. Smith
  7. Caiming Xiong
  8. Yoav Goldberg

In Computer Vision and Pattern Recognition

  1. Dacheng Tao
  2. Luc Van Gool
  3. Mohit Bansal
  4. Trevor Darrell
  5. Xiaogang Wang
  6. Ming-Hsuan Yang
  7. Bernt Schiele
  8. Yi Yang

In Graph Learning

  1. Shirui Pan
  2. Nicola De Cao
  3. Marco Damonte
  4. Shay B. Cohen
  5. Maosong Sun
  6. Michael Sejr Schlichtkrull
  7. Ivan Titov
  8. Philip S. Yu

In Machine Translation

  1. Daniel Beck
  2. Iryna Gurevych
  3. Ivan Vulic
  4. Goran Glavas
  5. Rui Wang
  6. Trevor Cohn
  7. Miguel Ballesteros
  8. Xiaodong He

Citations in the corpus