Coverage Embedding Models for Neural Machine Translation

By Haitao Mi, Baskaran Sankaran, Zhiguo Wang, Abe Ittycheriah. 2016

159 citations recorded by the source corpus.

In this paper, we enhance the attention-based neural machine translation (NMT) by adding explicit coverage embedding models to alleviate issues of repeating and dropping translations in NMT. For each source word, our model starts with a full coverage embedding vector to track the coverage status, and then keeps updating it with neural networks as the translation goes. Experiments on the large-scale Chinese-to-English task show that our enhanced model improves the translation quality significantly on various test sets over the strong large vocabulary NMT system.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Get To The Point: Summarization with Pointer-Generator Networks (2017, methodology): “Originating from Statistical Machine Translation (Koehn, 2009), coverage was adapted for NMT by Tu et al. (2016) and Mi et al. (2016), who both use a GRU to update the coverage vector each step.”
Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization (2016, methodology): “Note here that our method never requires r̃j[m] ≤ 0 (or r̃j [m] = 0) for all m at the last decoding time step j, as is generally required in the coverage Tu et al. (2016); Mi et al. (2016); Wu et al. (2016).”
Towards Decoding as Continuous Optimisation in Neural Machine Translation (2017, methodology): “These constraints have only been used for training in the previous works (Cohn et al., 2016; Mi et al., 2016).”
Unpaired Image Captioning by Language Pivoting (2018, background): “Recently, researchers have tried to improve the translation performance by introducing the attention mechanism [28, 44, 38, 12].”
Source-side Prediction for Neural Headline Generation (2017, background): “The coverage model [Mi et al., 2016, Tu et al., 2016] enforces the decoder to attend to every part of the source sequence to translate all semantic information in the source.”
Neural Machine Translation: A Review (2019, background): “An even tighter integration can be achieved by changing the NMT architecture itself and jointly training it with a coverage model [306, 310].”
Sparse and Constrained Attention for Neural Machine Translation (2018, background): “Other approaches involve architectural changes: providing coverage vectors to track the attention history (Mi et al., 2016; Tu et al., 2016), using gating architectures and adaptive attention to control the amount of source context provided ( Tu et al., 2017a; Li and Zhu, 2017), or adding a…”
Attention in Natural Language Processing (2019, methodology): “In [31], the representation is enhanced by making use of a subsymbolic representation for the coverage.”
Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder (2017, methodology): “, 2015) and the coverage model (Tu et al., 2016; Mi et al., 2016) provide effective mechanisms to control the generation of translation, these mechanisms work at the word level and cannot capture phrasal cohesion between the two languages (Fox, 2002; Kim et al.”
Neural Models of Text Normalization for Speech Applications (2019, background): “Tu et al. (2016) and Mi et al. (2016) argue that the root problem lies in the attention mechanism itself.”
Coupling Distributed and Symbolic Execution for Natural Language Queries (2016, background): “Our work sheds light on neural sequence prediction in general, for example, exploring word alignment (Mi et al., 2016) or chunking information (Zhou et al.”
RNN-based Encoder-decoder Approach with Word Frequency Estimation (2017, methodology): “Note here that our method never requires ˜ r j [ m ] ≤ 0 (or ˜ r j [ m ] = 0) for all m at the last decoding time step j , as is generally required in the coverage Tu et al. [2016], Mi et al. [2016], Wu et al. [2016].”
Towards Enhancing Faithfulness for Neural Machine Translation (2020, background): “(2016) and Mi et al. (2016) proposed a coverage mechanism to improve the accuracy of translation outputs.”