Knowledge Distillation for Bilingual Dictionary Induction

By Ndapandula Nakashole, Raphael Flauger. 2017

25 citations recorded by the source corpus.

Leveraging zero-shot learning to learn mapping functions between vector spaces of different languages is a promising approach to bilingual dictionary induction. However, methods using this approach have not yet achieved high accuracy on the task. In this paper, we propose a bridging approach, where our main contribution is a knowledge distillation training objective. As teachers, rich resource translation paths are exploited in this role. And as learners, translation paths involving low resource languages learn from the teachers. Our training objective allows seamless addition of teacher translation paths for any given low resource pair. Since our approach relies on the quality of monolingual word embeddings, we also propose to enhance vector representations of both the source and target language with linguistic information. Our experiments on various languages show large performance gains from our distillation training objective, obtaining as high as 17% accuracy improvements.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dacheng Tao
2. Jianping Gou
3. S. Maybank
4. Baosheng Yu
5. W. Zhang
6. Yuang Liu
7. Jun Wang
8. Chunyan Miao

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Jianping Gou
3. S. Maybank
4. Baosheng Yu
5. Yuang Liu
6. Chunyan Miao
7. W. Zhang
8. Jun Wang

Citations in the corpus

• Knowledge Distillation: A Survey (2020, background): “le-task teachers teach a multi-task student. For multilingual representations, knowledge distillation transferred knowledge among the multi-lingual word embeddings for bilingual dictionary induction (Nakashole and Flauger, 2017). The effectiveness of the knowledge transfer was also studied using ense”
• A Survey of Zero-Shot Learning (2019, background): “In bilingual dictionary induction [102, 105] and bilingual phrase table induction [28, 102] problems, from some seed bilingual word or phrase pairs, zero-shot learning helps to construct bilingual dictionaries or bilingual phrase tables.”
• Adaptive multi-teacher multi-level knowledge distillation (2020, background): “It has aroused some successful applications in computer vision [18, 29], natural language processing [9, 15], recommender system [23], etc.”