M. Patwary

Artificial Intelligence · 2019–2024

A researcher in the Science corpus: 11 papers; trusts 188 researchers and is trusted by 20 researchers here, as inferred from the citation graph.

Subfields

• Artificial Intelligence — 311 citations into it; #10,094 of 35,345 by trust here, #1,972 by citations
• Natural Language Processing — 206 citations into it; #4,207 of 5,688 by trust here, #2,203 by citations
• Computer Vision and Pattern Recognition — 123 citations into it
• Large Language Models — 106 citations into it
• Evaluation & Benchmarks — 62 citations into it
• Safety, Robustness & Interpretability — 39 citations into it
• Optimization & Training — 37 citations into it; #1,281 of 2,601 by trust here, #325 by citations
• Efficiency & Compression — 32 citations into it; #471 of 1,264 by trust here, #73 by citations

Whom M. Patwary trusts overall

Their strongest outbound trust, strongest first, inferred from their citations.

1. Diederik P. Kingma
2. Geoffrey E. Hinton
3. Ilya Sutskever
4. Yoshua Bengio
5. Alec Radford
6. Alex Krizhevsky
7. Kaiming He
8. Jimmy Ba
9. Yann LeCun
10. Quoc V. Le

Who trusts M. Patwary overall

1. Samyam Rajbhandari
2. Tri Dao
3. Devendra Singh Sachan
4. Dani Yogatama
5. Pengcheng He
6. Wayne Xin Zhao
7. Minjia Zhang
8. Michael Lewis
9. Junyi Li
10. Mohammad Shoeybi

Papers from M. Patwary's perspective

Ordered by how strongly the researchers M. Patwary trusts endorse each paper. Same corpus, a different order for every researcher.

1. Deep Residual Learning for Image Recognition (Computer Vision and Pattern Recognition)
2. ImageNet classification with deep convolutional neural networks
3. Attention is All you Need (Neural Information Processing Systems)
4. Adam: A Method for Stochastic Optimization (International Conference on Learning Representations)
5. Long Short-Term Memory
6. ImageNet: A large-scale hierarchical image database
7. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (North American Chapter of the Association for Computational Linguistics)
8. Gradient-based learning applied to document recognition
9. Sequence to Sequence Learning with Neural Networks (Neural Information Processing Systems)
10. Neural Machine Translation by Jointly Learning to Align and Translate (International Conference on Learning Representations)

Their papers

• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism (2019) — 2,986 citations
• Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022) — 859 citations
• Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM (2021) — 1,287 citations
• Factuality Enhanced Language Models for Open-Ended Text Generation (2022) — 310 citations
• Controllable Story Generation with External Knowledge Using Large-Scale Language Models (2020) — 140 citations
• Bio-Megatron: Larger Biomedical Domain Language Model (2020) — 177 citations
• StarCoder 2 and The Stack v2: The Next Generation (2024) — 698 citations
• Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models (2022) — 90 citations
• End-to-End Training of Neural Retrievers for Open-Domain Question Answering (2021) — 110 citations
• Efficient Large-Scale Language Model Training on GPU Clusters (2021) — 108 citations
• Training Question Answering Models from Synthetic Data (2020) — 179 citations