Training Deep Networks without Learning Rates Through Coin Betting

126 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates (2019, methodology): “We compare our methods against Adam [39], which is the most common adaptive method, and other methods that report better performance than Adam: coin-betting [60], L41 [68], and Adabound [51].”
A Modern Introduction to Online Learning (2019, methodology): “Recently, the same approach with a special coin-betting algorithm was also used for optimization of deep neural networks [Orabona and Tommasi, 2017].”
Estimating means of bounded random variables by betting (2020, methodology): “5 Online Newton Step (ONS-m) Betting algorithms play an essential role in online learning as several optimization problems can be framed in terms of coin-betting games [47, 48, 49, 23].”
Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions (2019, methodology): “The implementation used in this research is the default implementation of peephole connections in the Tensorflow framework.”