A survey of Algorithms and Analysis for Adaptive Online Learning

By H. Brendan McMahan. 2014

216 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

In Management Science and Operations Research

Citations in the corpus

Practical and Private (Deep) Learning without Sampling or Shuffling (2021, methodology): “In this section, we use the equivalence of non-private SGD and FTRL (McMahan, 2017) to establish equivalence between a variant of noisy-SGD and DP-FTRL, and hence make DP-SGD and DP-FTRl comparable.”
A unified view of entropy-regularized Markov decision processes (2017, methodology): “We next study algorithms arising from the Dual Averaging scheme (Xiao, 2010; McMahan, 2014), commonly known as Follow-the-Regularized-Leader in online learning (Shalev-Shwartz, 2012; Hazan et al., 2016).”
Adaptive and Universal Algorithms for Variational Inequalities with Optimal Convergence (2020, methodology): “Moreover, the prior algorithms use the off-by-one iterate (McMahan 2017), which is un-avoidable due to the use of the iterate movement in the step size.”
End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things (2020, background): “FL is a well-known distributed learning technique [9], [10].”
A Modern Introduction to Online Learning (2019, background): “3 History Bits The lower bound for OCO is quite standard, the proof presented is a simplified version of the one in Orabona and Pál [2018]. On the other hand, both the online learning literature and the optimization one almost ignored the issue of lower bounds for the unconstrained case.”
Convergence of Value Aggregation for Imitation Learning (2018, background): “The proof is based on a basic perturbation lemma in convex analysis (Lemma 4), which for example can be found in (McMahan, 2014), and a lemma for online learning (Lemma 5).”
AUC Maximization in the Era of Big Data and AI: A Survey (2022, background): “They develop a generalized Follow-The-Regularized-Leader framework [111] for AUC maximization with a lazy update which only involves a per-iteration cost O (k ).”
Variants of RMSProp and Adagrad with Logarithmic Regret Bounds (2017, background): “It is known that such bounds can be much better in practice than data independent bounds (Hazan et al., 2007),(McMahan, 2014).”
Predictor-Corrector Policy Optimization (2018, methodology): “To connect FTRL with mirror descent, we can roughly think of 〈(wg)1:n−1, ·〉 + ∑n m=1Brm as BRn in (9), since both act as regularization.”