QUality Estimation from ScraTCH (QUETCH): Deep Learning for Word-level Translation Quality Estimation

By Julia Kreutzer, Shigehiko Schamoni, Stefan Riezler. 2015

60 citations recorded by the source corpus.

This paper describes the system submitted by the University of Heidelberg to the Shared Task on Word-level Quality Estimation at the 2015 Workshop on Statistical Machine Translation. The submitted system combines a continuous space deep neural network, that learns a bilingual feature representation from scratch, with a linear combination of the manually defined baseline features provided by the task organizers. A combination of these orthogonal information sources shows significant improvements over the combined systems, and produces very competitive F1-scores for predicting word-level translation quality.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Findings of the 2016 Conference on Machine Translation (2016, methodology): “UNBABEL 2 ensemble uses a stacked architecture, inspired by the last year’s QUETCH+ system (Kreutzer et al., 2015), which combines three neural systems: one feedforward and two recurrent ones.”
Findings of the 2015 Workshop on Statistical Machine Translation (2015, neutral): “, 2015) HDCL Heidelberg University, Germany (Kreutzer et al., 2015) LORIA Lorraine Laboratory of Research in Computer Science and its Applications, France (Langlois, 2015) RTM-DCU Dublin City University, Ireland (Bicici et al.”
Findings of the 2017 Conference on Machine Translation (WMT17) (2017, background): “In addition to that, 6 new features were included which contain combinations of other features, and which proved useful in (Kreutzer et al., 2015; Martins et al., 2016): • Target word + left context.”
Recurrent Neural Network based Translation Quality Estimation (2016, methodology): “Kreutzer et al. (2015) apply neural networks using pre-trained alignments and word lookup-table to word-level QE, which achieve the excellent performance by using the combination of baseline features at word level.”
Unbabel’s Participation in the WMT16 Word-Level Translation Quality Estimation Shared Task (2016, methodology): “• We implemented three different neural systems, one extension of the original QUETCH model and two recurrent models with different depth.”
Learning What’s Easy: Fully Differentiable Neural Easy-First Taggers (2017, methodology): “We followed prior work (Kreutzer et al., 2015) and upweighted the BAD words in the loss function to make the model more pessimistic; we used a weight of 5 (tuned in the validation set).”
Ensembling Factored Neural Machine Translation Models for Automatic Post-Editing and Quality Estimation (2017, background): “Previous work has shown that alignment information between source and target is a critical component of current state-of-the-art word level QE systems (Kreutzer et al., 2015; Martins et al., 2016).”
MARMOT: A Toolkit for Translation Quality Estimation at the Word Level (2016, methodology): “The most widely used training models are Conditional Random Fields (CRF) (Luong et al., 2014; Shah et al., 2015) or neural networks: feed-forward (Kreutzer et al., 2015) as well as recurrent (Camargo de Souza et al., 2014).”
Pushing the Limits of Translation Quality Estimation (2017, methodology): “We train the model with the RMSProp algorithm (Tieleman and Hinton, 2012) by minimizing the cross-entropy with a linear penalty for BAD word predictions, as in Kreutzer et al. (2015).”