Robust Neural Machine Translation with Joint Textual and Phonetic Embedding
By Hairong Liu, Mingbo Ma, Liang Huang, Hao Xiong, Zhongjun He. 2018
60 citations recorded by the source corpus.
Neural machine translation (NMT) is notoriously sensitive to noises, but noises are almost inevitable in practice. One special kind of noise is the homophone noise, where words are replaced by other words with similar pronunciations. We propose to improve the robustness of NMT to homophone noises by 1) jointly embedding both textual and phonetic information of source sentences, and 2) augmenting the training dataset with homophone noises. Interestingly, to achieve better translation quality and more robustness, we found that most (though not all) weights should be put on the phonetic rather than textual information. Experiments show that our method not only significantly improves the robustness of NMT to homophone noises, but also surprisingly improves the translation quality on some clean test sets.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Luke S. Zettlemoyer
- Sebastian Riedel
- Mohit Bansal
- Hannaneh Hajishirzi
- Michael Lewis
- Wen-tau Yih
- Yejin Choi
- Jianfeng Gao
In Natural Language Processing
- Sebastian Riedel
- Mohit Bansal
- Luke S. Zettlemoyer
- Douwe Kiela
- Wen-tau Yih
- Yejin Choi
- Jason Aaron Edward Weston
- Graham Neubig
Citations in the corpus
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020, methodology): “For 2-way classification, we compare against the model from [51], which trains RoBERTa [31] to classify the claim as true or false given the gold evidence sentence.”
- Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections (2021, methodology): “To meta-tune non-Seq2Seq pretrained models, such as BERT (Devlin et al., 2019) or RoBERTa (Liu et al., 2019), we add an MLP layer on top of the pooled output/“[CLS]" token to classify between “Yes"/“No".”
- Robust Neural Machine Translation with Doubly Adversarial Inputs (2019, background): “Liu et al. (2018) examined the homophonic noise for Chinese translation.”