Model Architectures

Transformers, attention, recurrent/convolutional nets, state-space and sequence models.

A topic in the Science corpus, drawn from 1,688 papers.

Trusted voices in Model Architectures

Ordered by the trust Model Architectures's own network concentrates on each researcher, most first.

  1. Kaiming He — trusted by 8,294 researchers here
  2. Shaoqing Ren — trusted by 7,208 researchers here
  3. X. Zhang — trusted by 7,120 researchers here
  4. Karen Simonyan — trusted by 6,453 researchers here
  5. Andrew Zisserman — trusted by 5,849 researchers here
  6. Yoshua Bengio — trusted by 5,388 researchers here
  7. Christian Szegedy — trusted by 5,408 researchers here
  8. Sergey Ioffe — trusted by 4,885 researchers here
  9. J. Urgen Schmidhuber — trusted by 4,531 researchers here
  10. Geoffrey E. Hinton — trusted by 4,440 researchers here
  11. Sepp Hochreiter — trusted by 4,103 researchers here
  12. Quoc V. Le — trusted by 3,449 researchers here

Most-cited papers in Model Architectures

Ordered by the recorded citation count from the source corpus; a citation is not an endorsement.

  1. Deep Residual Learning for Image Recognition (2015, Computer Vision and Pattern Recognition) — 234,194 citations
  2. Attention is All you Need (2017, Neural Information Processing Systems) — 185,508 citations
  3. Random Forests (2001) — 119,944 citations
  4. Very Deep Convolutional Networks for Large-Scale Image Recognition (2014, International Conference on Learning Representations) — 112,917 citations
  5. U-Net: Convolutional Networks for Biomedical Image Segmentation (2015, International Conference on Medical Image Computing and Computer-Assisted Intervention) — 98,692 citations
  6. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2020, International Conference on Learning Representations) — 66,490 citations
  7. Going deeper with convolutions (2014) — 47,651 citations
  8. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (2015, International Conference on Machine Learning) — 47,277 citations
  9. Densely Connected Convolutional Networks (2016) — 44,055 citations
  10. Squeeze-and-Excitation Networks (2017) — 35,229 citations
  11. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows (2021) — 33,967 citations
  12. Mask R-CNN (2017) — 32,519 citations
  13. Rethinking the Inception Architecture for Computer Vision (2015) — 31,459 citations
  14. Neural Machine Translation by Jointly Learning to Align and Translate (2014, International Conference on Learning Representations) — 29,733 citations
  15. Graph Attention Networks (2017, International Conference on Learning Representations) — 27,282 citations
  16. CBAM: Convolutional Block Attention Module (2018) — 24,594 citations
  17. Sequence to Sequence Learning with Neural Networks (2014, Neural Information Processing Systems) — 22,211 citations
  18. End-to-End Object Detection with Transformers (2020, European Conference on Computer Vision) — 19,102 citations
  19. Rectified Linear Units Improve Restricted Boltzmann Machines (2010) — 18,880 citations
  20. Xception: Deep Learning with Depthwise Separable Convolutions (2016) — 18,118 citations
  21. A Fast Learning Algorithm for Deep Belief Nets (2006) — 16,970 citations
  22. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning (2016) — 15,641 citations
  23. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling (2014) — 14,897 citations
  24. Caffe: Convolutional Architecture for Fast Feature Embedding (2014) — 14,879 citations
  25. 5分で分かる!? 有名論文ナナメ読み:Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding (2020) — 14,205 citations
  26. Aggregated Residual Transformations for Deep Neural Networks (2016) — 11,783 citations
  27. Identity Mappings in Deep Residual Networks (2016) — 11,282 citations
  28. Non-local Neural Networks (2017) — 10,287 citations
  29. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting (2015) — 9,818 citations
  30. Multi-Scale Context Aggregation by Dilated Convolutions (2015) — 9,514 citations
  31. Wide Residual Networks (2016) — 8,993 citations
  32. Deep Sparse Rectifier Neural Networks (2011) — 8,901 citations
  33. A ConvNet for the 2020s (2022) — 8,792 citations
  34. Effective Approaches to Attention-based Neural Machine Translation (2015) — 8,424 citations
  35. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation (2016) — 8,190 citations
  36. Spatial Transformer Networks (2015) — 8,156 citations
  37. Mamba: Linear-Time Sequence Modeling with Selective State Spaces (2023, arXiv.org) — 8,131 citations
  38. Improving neural networks by preventing co-adaptation of feature detectors (2012) — 8,038 citations
  39. Rectifier Nonlinearities Improve Neural Network Acoustic Models (2013) — 7,685 citations
  40. On the Properties of Neural Machine Translation: Encoder–Decoder Approaches (2014) — 7,554 citations
  41. Learning to Forget: Continual Prediction with LSTM (2000) — 7,541 citations
  42. Neural Ordinary Differential Equations (2018) — 7,349 citations
  43. Gaussian Error Linear Units (GELUs) (2016) — 7,098 citations
  44. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling (2018) — 6,867 citations
  45. Network In Network (2013) — 6,739 citations
  46. Scalable Diffusion Models with Transformers (2022, IEEE International Conference on Computer Vision) — 6,663 citations
  47. Long-term recurrent convolutional networks for visual recognition and description (2014) — 6,439 citations
  48. LSTM: A Search Space Odyssey (2015) — 6,287 citations
  49. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks (2019) — 6,251 citations
  50. Learning Transferable Architectures for Scalable Image Recognition (2017) — 6,216 citations

Most active researchers

By papers in this topic: a volume measure, not trust.