Model Architectures
Transformers, attention, recurrent/convolutional nets, state-space and sequence models.
A topic in the Science corpus, drawn from 1,688 papers.
Trusted voices in Model Architectures
Ordered by the trust Model Architectures's own network concentrates on each researcher, most first.
- Kaiming He — trusted by 8,294 researchers here
- Shaoqing Ren — trusted by 7,208 researchers here
- X. Zhang — trusted by 7,120 researchers here
- Karen Simonyan — trusted by 6,453 researchers here
- Andrew Zisserman — trusted by 5,849 researchers here
- Yoshua Bengio — trusted by 5,388 researchers here
- Christian Szegedy — trusted by 5,408 researchers here
- Sergey Ioffe — trusted by 4,885 researchers here
- J. Urgen Schmidhuber — trusted by 4,531 researchers here
- Geoffrey E. Hinton — trusted by 4,440 researchers here
- Sepp Hochreiter — trusted by 4,103 researchers here
- Quoc V. Le — trusted by 3,449 researchers here
Most-cited papers in Model Architectures
Ordered by the recorded citation count from the source corpus; a citation is not an endorsement.
- Deep Residual Learning for Image Recognition (2015, Computer Vision and Pattern Recognition) — 234,194 citations
- Attention is All you Need (2017, Neural Information Processing Systems) — 185,508 citations
- Random Forests (2001) — 119,944 citations
- Very Deep Convolutional Networks for Large-Scale Image Recognition (2014, International Conference on Learning Representations) — 112,917 citations
- U-Net: Convolutional Networks for Biomedical Image Segmentation (2015, International Conference on Medical Image Computing and Computer-Assisted Intervention) — 98,692 citations
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2020, International Conference on Learning Representations) — 66,490 citations
- Going deeper with convolutions (2014) — 47,651 citations
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (2015, International Conference on Machine Learning) — 47,277 citations
- Densely Connected Convolutional Networks (2016) — 44,055 citations
- Squeeze-and-Excitation Networks (2017) — 35,229 citations
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021) — 33,967 citations
- Mask R-CNN (2017) — 32,519 citations
- Rethinking the Inception Architecture for Computer Vision (2015) — 31,459 citations
- Neural Machine Translation by Jointly Learning to Align and Translate (2014, International Conference on Learning Representations) — 29,733 citations
- Graph Attention Networks (2017, International Conference on Learning Representations) — 27,282 citations
- CBAM: Convolutional Block Attention Module (2018) — 24,594 citations
- Sequence to Sequence Learning with Neural Networks (2014, Neural Information Processing Systems) — 22,211 citations
- End-to-End Object Detection with Transformers (2020, European Conference on Computer Vision) — 19,102 citations
- Rectified Linear Units Improve Restricted Boltzmann Machines (2010) — 18,880 citations
- Xception: Deep Learning with Depthwise Separable Convolutions (2016) — 18,118 citations
- A Fast Learning Algorithm for Deep Belief Nets (2006) — 16,970 citations
- Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning (2016) — 15,641 citations
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling (2014) — 14,897 citations
- Caffe: Convolutional Architecture for Fast Feature Embedding (2014) — 14,879 citations
- 5分で分かる!? 有名論文ナナメ読み:Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding (2020) — 14,205 citations
- Aggregated Residual Transformations for Deep Neural Networks (2016) — 11,783 citations
- Identity Mappings in Deep Residual Networks (2016) — 11,282 citations
- Non-local Neural Networks (2017) — 10,287 citations
- Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting (2015) — 9,818 citations
- Multi-Scale Context Aggregation by Dilated Convolutions (2015) — 9,514 citations
- Wide Residual Networks (2016) — 8,993 citations
- Deep Sparse Rectifier Neural Networks (2011) — 8,901 citations
- A ConvNet for the 2020s (2022) — 8,792 citations
- Effective Approaches to Attention-based Neural Machine Translation (2015) — 8,424 citations
- 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation (2016) — 8,190 citations
- Spatial Transformer Networks (2015) — 8,156 citations
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces (2023, arXiv.org) — 8,131 citations
- Improving neural networks by preventing co-adaptation of feature detectors (2012) — 8,038 citations
- Rectifier Nonlinearities Improve Neural Network Acoustic Models (2013) — 7,685 citations
- On the Properties of Neural Machine Translation: Encoder–Decoder Approaches (2014) — 7,554 citations
- Learning to Forget: Continual Prediction with LSTM (2000) — 7,541 citations
- Neural Ordinary Differential Equations (2018) — 7,349 citations
- Gaussian Error Linear Units (GELUs) (2016) — 7,098 citations
- An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling (2018) — 6,867 citations
- Network In Network (2013) — 6,739 citations
- Scalable Diffusion Models with Transformers (2022, IEEE International Conference on Computer Vision) — 6,663 citations
- Long-term recurrent convolutional networks for visual recognition and description (2014) — 6,439 citations
- LSTM: A Search Space Odyssey (2015) — 6,287 citations
- ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks (2019) — 6,251 citations
- Learning Transferable Architectures for Scalable Image Recognition (2017) — 6,216 citations
Most active researchers
By papers in this topic: a volume measure, not trust.
- Yoshua Bengio — 59 papers
- J. Urgen Schmidhuber — 29 papers
- Quoc V. Le — 27 papers
- Geoffrey E. Hinton — 21 papers
- Oriol Vinyals — 20 papers
- Ruslan Salakhutdinov — 18 papers
- Yann LeCun — 17 papers
- Kyunghyun Cho — 16 papers
- Alex Graves — 15 papers
- Razvan Pascanu — 15 papers
- Yi Tay — 13 papers
- Jianfeng Gao — 13 papers