Samyam Rajbhandari

Artificial Intelligence · 2017–2022

A researcher in the Science corpus: 12 papers; trusts 87 researchers and is trusted by 41 researchers here, as inferred from the citation graph.

Subfields

• Artificial Intelligence — 333 citations into it; #12,577 of 35,345 by trust here, #2,388 by citations
• Natural Language Processing — 152 citations into it
• Computer Vision and Pattern Recognition — 143 citations into it; #13,833 of 21,575 by trust here, #2,494 by citations
• Large Language Models — 128 citations into it
• Efficiency & Compression — 100 citations into it; #195 of 1,264 by trust here, #56 by citations
• Optimization & Training — 79 citations into it; #2,473 of 2,601 by trust here, #544 by citations
• Model Architectures — 50 citations into it
• General Social Sciences — 17 citations into it

Whom Samyam Rajbhandari trusts overall

Their strongest outbound trust, strongest first, inferred from their citations.

1. Diederik P. Kingma
2. Geoffrey E. Hinton
3. Noam Shazeer
4. Alec Radford
5. Yoshua Bengio
6. Jimmy Ba
7. Alex Krizhevsky
8. Ilya Sutskever
9. Kaiming He
10. Sepp Hochreiter

Who trusts Samyam Rajbhandari overall

1. Vijay Anand Korthikanti
2. D. Narayanan
3. William Fedus
4. Reza Yazdani Aminabadi
5. Jiarui Fang
6. Can Bülent Karakus
7. Jared Casper
8. Zhuohan Li
9. Yuxin Fang
10. Yuxian Gu

Papers from Samyam Rajbhandari's perspective

Ordered by how strongly the researchers Samyam Rajbhandari trusts endorse each paper. Same corpus, a different order for every researcher.

1. Deep Residual Learning for Image Recognition (Computer Vision and Pattern Recognition)
2. ImageNet classification with deep convolutional neural networks
3. Attention is All you Need (Neural Information Processing Systems)
4. Adam: A Method for Stochastic Optimization (International Conference on Learning Representations)
5. Long Short-Term Memory
6. ImageNet: A large-scale hierarchical image database
7. Gradient-based learning applied to document recognition
8. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (North American Chapter of the Association for Computational Linguistics)
9. Very Deep Convolutional Networks for Large-Scale Image Recognition (International Conference on Learning Representations)
10. ImageNet Large Scale Visual Recognition Challenge

Their papers

• Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022) — 859 citations
• ZeRO-Offload: Democratizing Billion-Scale Model Training (2021) — 618 citations
• DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters (2020) — 2,160 citations
• Scalable and Efficient MoE Training for Multitask Multilingual Models (2021) — 101 citations
• ZeRO: Memory optimizations Toward Training Trillion Parameter Models (2019) — 1,913 citations
• ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep learning (2021) — 548 citations
• DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale (2022) — 591 citations
• DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale (2022) — 640 citations
• 1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed (2021) — 111 citations
• Learning Intrinsic Sparse Structures within Long Short-term Memory (2017) — 144 citations
• BLOOM: A 176B-Parameter Open-Access Multilingual Language Model (2022) — 2,975 citations
• ZeRO: Memory Optimization Towards Training A Trillion Parameter Models (2019) — 616 citations