MLPruning: A Multilevel Structured Pruning Framework for Transformer-based Models

By Zi-Mei Yao, Linjian Ma, Sheng Shen, Kurt Keutzer, Michael W. Mahoney. 2021

14 citations recorded by the source corpus.

Subfields: Model Architectures, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

1. A. Gholami
2. Kurt Keutzer
3. Sehoon Kim
4. Xiaoxia Wu
5. Sheng Shen
6. Yuxiong He
7. Conglong Li
8. Reza Yazdani Aminabadi

In Efficiency & Compression

1. Kurt Keutzer
2. A. Gholami
3. Zi-Mei Yao
4. Sehoon Kim
5. Sheng Shen
6. Minjia Zhang
7. Yuxiong He
8. Conglong Li

Citations in the corpus

• ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers (2022, background): “Model compression has been explored from different aspects [25, 37, 39, 34, 43, 20, 24, 50, 18, 74, 40, 26, 55, 59, 28, 60, 68, 33, 14, 38, 31].”
• Learned Token Pruning for Transformers (2021, methodology): “Recent block pruning schemes chunk weight matrices into multiple blocks and prune them based on group Lasso optimization [26], adaptive regularization [53], and movement pruning [24].”
• Extreme Compression for Pre-trained Transformers Made Simple and Efficient (2022, background): “Many research efforts have been made to compress these huge transformer models including knowledge distillation [20, 54, 53, 61], pruning [57, 43, 8], and low-rank decomposition [30].”