Fully Quantized Transformer for Machine Translation

By Gabriele Prato, Ella Charlaix, Mehdi Rezagholizadeh. 2019

72 citations recorded by the source corpus.

State-of-the-art neural machine translation methods employ massive amounts of parameters. Drastically reducing computational costs of such methods without affecting performance has been up to this point unsuccessful. To this end, we propose FullyQT: an all-inclusive quantization strategy for the Transformer. To the best of our knowledge, we are the first to show that it is possible to avoid any loss in translation quality with a fully quantized Transformer. Indeed, compared to full-precision, our 8-bit models score greater or equal BLEU on most tasks. Comparing ourselves to all previously proposed methods, we achieve state-of-the-art quantization results.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qun Liu
2. Dacheng Tao
3. Xin Jiang
4. Yunhe Wang
5. Lifeng Shang
6. Wei Zhang
7. Kai Han
8. Chunjing Xu

In Computer Vision and Pattern Recognition

1. Yunhe Wang
2. Kai Han
3. Dacheng Tao
4. Jianyuan Guo
5. Chunjing Xu
6. Qun Liu
7. Wei Zhang
8. Yehui Tang

In Efficiency & Compression

1. Lu Hou
2. Qun Liu
3. Lifeng Shang
4. Xin Jiang
5. Wei Zhang
6. Mehdi Rezagholizadeh
7. Yunhe Wang
8. Kai Han

In Machine Translation

1. Dacheng Tao
2. Qun Liu
3. Xin Jiang

Citations in the corpus

• Sparse is Enough in Scaling Transformers (2021, methodology): “Our approach is complementary to model quantization studies [35, 38, 28] that use fewer bits for the weights.”
• TernaryBERT: Distillation-aware Ultra-low Bit BERT (2020, methodology): “In (Prato et al., 2019; Zafrir et al., 2019), 8-bit quantization is successfully applied to Transformer-based models with comparable performance as the full-precision baseline.”
• DyLoRA: Parameter-Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation (2022, methodology): “Inspired by the idea of nested dropout (Rippel et al., 2014), we pursue ordering the representations of the bottle-neck at the low-rank adapter modules with a new recipe.”
• A Survey on Vision Transformer (2020, methodology): “Quantization FullyQT [97] Fully quantized transformer EMNLP Findings 2020”