Optimization & Training
Optimizers, normalization, regularization, and training dynamics/techniques.
A topic in the Science corpus, drawn from 2,103 papers.
Trusted voices in Optimization & Training
Ordered by the trust Optimization & Training's own network concentrates on each researcher, most first.
- Jimmy Ba — trusted by 10,592 researchers here
- Diederik P. Kingma — trusted by 10,281 researchers here
- Kaiming He — trusted by 6,781 researchers here
- Shaoqing Ren — trusted by 6,656 researchers here
- X. Zhang — trusted by 6,606 researchers here
- Yoshua Bengio — trusted by 3,728 researchers here
- Sergey Ioffe — trusted by 3,763 researchers here
- Christian Szegedy — trusted by 3,749 researchers here
- Robert Tibshirani — trusted by 2,502 researchers here
- Geoffrey E. Hinton — trusted by 3,256 researchers here
- Yurii E. Nesterov — trusted by 1,820 researchers here
- Frank Hutter — trusted by 2,107 researchers here
Most-cited papers in Optimization & Training
Ordered by the recorded citation count from the source corpus; a citation is not an endorsement.
- Deep Residual Learning for Image Recognition (2015, Computer Vision and Pattern Recognition) — 234,194 citations
- Adam: A Method for Stochastic Optimization (2014, International Conference on Learning Representations) — 168,773 citations
- XGBoost: A Scalable Tree Boosting System (2016) — 57,634 citations
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (2015, International Conference on Machine Learning) — 47,277 citations
- Dropout: a simple way to prevent neural networks from overfitting (2014) — 43,754 citations
- Convex Optimization (2010) — 40,693 citations
- Decoupled Weight Decay Regularization (2017) — 36,546 citations
- SMOTE: Synthetic Minority Over-sampling Technique (2002) — 31,281 citations
- Proximal Policy Optimization Algorithms (2017, arXiv.org) — 29,461 citations
- Understanding the difficulty of training deep feedforward neural networks (2010) — 19,426 citations
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium (2017) — 18,997 citations
- Numerical optimization (2004) — 17,888 citations
- LightGBM: A Highly Efficient Gradient Boosting Decision Tree (2017) — 15,550 citations
- A tutorial on support vector regression (2004) — 12,903 citations
- Layer Normalization (2016) — 12,877 citations
- Ridge Regression: Biased Estimation for Nonorthogonal Problems (2000) — 12,604 citations
- mixup: Beyond Empirical Risk Minimization (2017) — 12,076 citations
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (2016) — 11,798 citations
- Adaptive Subgradient Methods for Online Learning and Stochastic Optimization (2011) — 11,425 citations
- Identity Mappings in Deep Residual Networks (2016) — 11,282 citations
- Random Search for Hyper-Parameter Optimization (2012) — 11,272 citations
- Improved Training of Wasserstein GANs (2017) — 11,031 citations
- SGDR: Stochastic Gradient Descent with Warm Restarts (2016) — 10,851 citations
- Addendum: Regularization and variable selection via the elastic net (2005) — 10,689 citations
- Improved Techniques for Training GANs (2016) — 10,497 citations
- Overcoming catastrophic forgetting in neural networks (2016) — 10,384 citations
- Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (2003) — 9,804 citations
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties (2001) — 9,697 citations
- Practical Bayesian Optimization of Machine Learning Algorithms (2012) — 9,594 citations
- Algorithms for Non-negative Matrix Factorization (2000) — 9,381 citations
- Federated Optimization in Heterogeneous Networks (2018, Conference on Machine Learning and Systems) — 8,339 citations
- Trust Region Policy Optimization (2015, International Conference on Machine Learning) — 8,139 citations
- Improving neural networks by preventing co-adaptation of feature detectors (2012) — 8,038 citations
- Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers (2011) — 8,037 citations
- Stable signal recovery from incomplete and inaccurate measurements (2005) — 7,765 citations
- Optimal Transport: Old and New (2008) — 7,666 citations
- Curriculum learning (2009) — 7,325 citations
- Addressing Function Approximation Error in Actor-Critic Methods (2018) — 7,119 citations
- A Practical Guide to Support Vector Classication (2008) — 7,108 citations
- An overview of gradient descent optimization algorithms (2016) — 6,991 citations
- ADADELTA: An Adaptive Learning Rate Method (2012) — 6,871 citations
- © Institute of Mathematical Statistics, 2004 LEAST ANGLE REGRESSION — 6,720 citations
- Stochastic gradient boosting (2002) — 6,637 citations
- Introductory Lectures on Convex Optimization - A Basic Course (2014) — 6,525 citations
- Special Invited Paper-Additive logistic regression: A statistical view of boosting (2000) — 6,512 citations
- Large-Scale Machine Learning with Stochastic Gradient Descent (2010) — 6,396 citations
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) (2015) — 6,140 citations
- CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features (2019) — 6,079 citations
- Neural Architecture Search with Reinforcement Learning (2016) — 5,967 citations
- Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories (2004) — 5,814 citations
Most active researchers
By papers in this topic: a volume measure, not trust.
- Yoshua Bengio — 44 papers
- Peter Richtárik — 36 papers
- Francis R. Bach — 34 papers
- Simon Lacoste-Julien — 32 papers
- Nati Srebro — 25 papers
- Martin Jaggi — 22 papers
- Michael I. Jordan — 20 papers
- L. Eon Bottou — 18 papers
- Mark W. Schmidt — 17 papers
- Yann LeCun — 17 papers
- Tong Zhang — 17 papers
- Shai Shalev-Shwartz — 16 papers