Bilingually-constrained Phrase Embeddings for Machine Translation

By Jiajun Zhang, Shujie Liu, Mu Li, M. Zhou, Chengqing Zong. 2014

120 citations recorded by the source corpus.

We propose Bilingually-constrained Recursive Auto-encoders (BRAE) to learn semantic phrase embeddings (compact vector representations for phrases), which can distinguish the phrases with different semantic meanings. The BRAE is trained in a way that minimizes the semantic distance of translation equivalents and maximizes the semantic distance of nontranslation pairs simultaneously. After training, the model learns how to embed each phrase semantically in two languages and also learns how to transform semantic embedding space in one language to the other. We evaluate our proposed method on two end-to-end SMT tasks (phrase table pruning and decoding with phrasal semantic similarities) which need to measure semantic similarity between a source phrase and its translation candidates. Extensive experiments show that the BRAE is remarkably effective in these two tasks.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Li Deng
3. Mohit Bansal
4. Xiaodong He
5. Hang Li
6. Dong Yu
7. Kyunghyun Cho
8. Yoshua Bengio

In Representation & Self-Supervised Learning

1. Jianfeng Gao
2. Xiaodong He
3. Hang Li
4. Li Deng
5. Zhengdong Lu
6. Qingcai Chen
7. Timothy Baldwin
8. Jianshu Chen

In Machine Translation

1. Li Deng
2. Jianfeng Gao
3. Xiaodong He
4. Yelong Shen
5. Chris Dyer
6. Bo Xu
7. Po-Yao (Bernie) Huang
8. Chengqing Zong

Citations in the corpus

• Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval (2015, methodology): “In [14], a Bilingually-constrained Recursive Auto-encoders (BRAE) is proposed to create semantic vector representation for phrases.”
• Attention-based Multimodal Neural Machine Translation (2016, methodology): “In machine translation fields, (Zhang et al., 2014; Su et al., 2015) learned phrase-level bilingual repre-”
• Abstractive Cross-Language Summarization via Translation Model Enhanced Predicate Argument Structure Fusing (2016, methodology): “In our future work, we will try other methods, such as measuring the translation confidence in phrase embedding space [13] or in tree structures [14].”
• Context-Dependent Translation Selection Using Convolutional Neural Network (2015, methodology): “• CICM (context-independent convolutional matching) model: Following the previous works (Gao et al., 2014; Zhang et al., 2014; Cho et al., 2014b), we calculate the matching degree of a phrase pair without considering any contextual information.”
• An Empirical Evaluation of doc2vec with Practical Insights into Document Embedding Generation (2016, methodology): “…of NLP applications, including initialising neural network models for objective visual recognition (Frome et al., 2013) or machine translation (Zhang et al., 2014; Li et al., 2014), as well as directly modelling word-to-word relationships (Mikolov et al., 2013a; Zhao et al., 2015; Salehi et…”
• End-to-End Language Identification Using Attention-Based Recurrent Neural Networks (2016, background): “Language category embedding converts a language gold label into a dense, fixed-size, real valued vector representation, which is similar in spirit of the word embedding [26, 27, 28, 29, 30].”