Yuxiong He

Artificial Intelligence · with Rajbhandari · 2017–2022

A researcher in the Science corpus: 14 papers; trusts 272 researchers and is trusted by 53 researchers here, as inferred from the citation graph.

Subfields

• Artificial Intelligence — 324 citations into it; #7,686 of 35,345 by trust here, #2,150 by citations
• Computer Vision and Pattern Recognition — 182 citations into it; #8,653 of 21,575 by trust here, #2,148 by citations
• Efficiency & Compression — 154 citations into it; #155 of 1,264 by trust here, #43 by citations
• Large Language Models — 116 citations into it
• Natural Language Processing — 115 citations into it
• Optimization & Training — 85 citations into it; #2,474 of 2,601 by trust here, #544 by citations
• Model Architectures — 59 citations into it
• Evaluation & Benchmarks — 51 citations into it

Whom Yuxiong He trusts overall

Their strongest outbound trust, strongest first, inferred from their citations.

1. Alex Krizhevsky
2. Geoffrey E. Hinton
3. Diederik P. Kingma
4. Yoshua Bengio
5. Ilya Sutskever
6. Kaiming He
7. Yann LeCun
8. Jimmy Ba
9. Sepp Hochreiter
10. Oriol Vinyals

Who trusts Yuxiong He overall

1. Samyam Rajbhandari
2. Wujie Wen
3. Vijay Anand Korthikanti
4. Huanrui Yang
5. Sehoon Kim
6. Elias Frantar
7. D. Narayanan
8. Zi-Mei Yao
9. Can Bülent Karakus
10. Yuxin Fang

Papers from Yuxiong He's perspective

Ordered by how strongly the researchers Yuxiong He trusts endorse each paper. Same corpus, a different order for every researcher.

1. Deep Residual Learning for Image Recognition (Computer Vision and Pattern Recognition)
2. ImageNet classification with deep convolutional neural networks
3. Adam: A Method for Stochastic Optimization (International Conference on Learning Representations)
4. Attention is All you Need (Neural Information Processing Systems)
5. Long Short-Term Memory
6. ImageNet: A large-scale hierarchical image database
7. Gradient-based learning applied to document recognition
8. Learning Multiple Layers of Features from Tiny Images
9. ImageNet Large Scale Visual Recognition Challenge
10. Going deeper with convolutions

Their papers

• Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022) — 859 citations
• ZeRO-Offload: Democratizing Billion-Scale Model Training (2021) — 618 citations
• DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters (2020) — 2,160 citations
• Scalable and Efficient MoE Training for Multitask Multilingual Models (2021) — 101 citations
• ZeRO: Memory optimizations Toward Training Trillion Parameter Models (2019) — 1,913 citations
• ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep learning (2021) — 548 citations
• HyperDrive: exploring hyperparameters with POP scheduling (2017) — 41 citations
• DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale (2022) — 591 citations
• ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers (2022) — 759 citations
• DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale (2022) — 640 citations
• 1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed (2021) — 111 citations
• Learning Intrinsic Sparse Structures within Long Short-term Memory (2017) — 144 citations
• Extreme Compression for Pre-trained Transformers Made Simple and Efficient (2022) — 31 citations
• ZeRO: Memory Optimization Towards Training A Trillion Parameter Models (2019) — 616 citations