Efficiency & Compression
Quantization, distillation, pruning, parameter-efficient fine-tuning (LoRA/adapters), efficient inference.
A topic in the Science corpus, drawn from 918 papers.
Trusted voices in Efficiency & Compression
Ordered by the trust Efficiency & Compression's own network concentrates on each researcher, most first.
- Geoffrey E. Hinton — trusted by 1,356 researchers here
- Oriol Vinyals — trusted by 1,194 researchers here
- Joseph Dean — trusted by 1,179 researchers here
- Song Han — trusted by 901 researchers here
- Quoc V. Le — trusted by 833 researchers here
- William (Bill) J. Dally — trusted by 811 researchers here
- Mingxing Tan — trusted by 802 researchers here
- Xiangyu Zhang — trusted by 589 researchers here
- Yoshua Bengio — trusted by 489 researchers here
- Huizi Mao — trusted by 472 researchers here
- Matthieu Courbariaux — trusted by 393 researchers here
- Andrew G. Howard — trusted by 421 researchers here
Most-cited papers in Efficiency & Compression
Ordered by the recorded citation count from the source corpus; a citation is not an endorsement.
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017) — 25,507 citations
- MobileNetV2: Inverted Residuals and Linear Bottlenecks (2018) — 25,197 citations
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (2019) — 25,169 citations
- Distilling the Knowledge in a Neural Network (2015, arXiv.org) — 25,160 citations
- LoRA: Low-Rank Adaptation of Large Language Models (2021, International Conference on Learning Representations) — 21,145 citations
- Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding (2015) — 10,226 citations
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter (2019) — 10,143 citations
- Searching for MobileNetV3 (2019) — 9,744 citations
- Training data-efficient image transformers & distillation through attention (2020) — 9,450 citations
- ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices (2017) — 8,533 citations
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size (2016) — 8,474 citations
- Learning both Weights and Connections for Efficient Neural Network (2015) — 7,826 citations
- ALBERT: A Lite BERT for Self-supervised Learning of Language Representations (2019) — 7,601 citations
- Enhanced Deep Residual Networks for Single Image Super-Resolution (2017) — 7,355 citations
- EfficientDet: Scalable and Efficient Object Detection (2019) — 7,168 citations
- Efficient Memory Management for Large Language Model Serving with PagedAttention (2023) — 7,142 citations
- Parameter-Efficient Transfer Learning for NLP (2019) — 6,877 citations
- ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) — 6,472 citations
- ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks (2019) — 6,251 citations
- Prefix-Tuning: Optimizing Continuous Prompts for Generation (2021) — 6,083 citations
- Federated Learning: Strategies for Improving Communication Efficiency (2016) — 5,508 citations
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer (2017) — 5,113 citations
- QLoRA: Efficient Finetuning of Quantized LLMs (2023, Neural Information Processing Systems) — 5,083 citations
- Bag of Tricks for Efficient Text Classification (2016) — 5,082 citations
- Coordinate Attention for Efficient Mobile Network Design (2021) — 5,022 citations
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (2022, Neural Information Processing Systems) — 4,798 citations
- FitNets: Hints for Thin Deep Nets (2014) — 4,770 citations
- XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks (2016) — 4,754 citations
- EfficientNetV2: Smaller Models and Faster Training (2021) — 4,600 citations
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference (2017) — 4,475 citations
- Knowledge Distillation: A Survey (2020) — 4,395 citations
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (2018) — 4,345 citations
- Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity (2021, Journal of machine learning research) — 4,336 citations
- Pruning Filters for Efficient ConvNets (2016) — 4,125 citations
- MnasNet: Platform-Aware Neural Architecture Search for Mobile (2018) — 3,454 citations
- Accelerating the Super-Resolution Convolutional Neural Network (2016) — 3,453 citations
- Towards Federated Learning at Scale: System Design (2019) — 3,255 citations
- BinaryConnect: Training Deep Neural Networks with binary weights during propagations (2015) — 3,245 citations
- Reformer: The Efficient Transformer (2020) — 3,006 citations
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism (2019) — 2,986 citations
- Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention (2020) — 2,953 citations
- Model compression (2006) — 2,875 citations
- Learning Efficient Convolutional Networks through Network Slimming (2017) — 2,811 citations
- Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks (2016) — 2,806 citations
- Channel Pruning for Accelerating Very Deep Neural Networks (2017) — 2,781 citations
- Regularization of Neural Networks using DropConnect (2013) — 2,649 citations
- Active Learning for Convolutional Neural Networks: A Core-Set Approach (2017) — 2,623 citations
- EIE: Efficient Inference Engine on Compressed Deep Neural Network (2016) — 2,611 citations
- Learning Structured Sparsity in Deep Neural Networks (2016) — 2,511 citations
- Vector Quantization (2017) — 2,461 citations
Most active researchers
By papers in this topic: a volume measure, not trust.
- Kurt Keutzer — 21 papers
- Song Han — 20 papers
- Quoc V. Le — 14 papers
- Joseph E. Gonzalez — 14 papers
- Yuxiong He — 12 papers
- Yoshua Bengio — 11 papers
- Dan Alistarh — 11 papers
- Samyam Rajbhandari — 10 papers
- Matei A. Zaharia — 9 papers
- Bryan Catanzaro — 9 papers
- Ion Stoica — 9 papers
- Ruoming Pang — 9 papers