Implicit Online Learning

By Brian Kulis, Peter L. Bartlett. 2010

99 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Protection Against Reconstruction and Its Applications in Private Federated Learning (2018, methodology): “An alternative is to use stochastic proximal-point-type updates [10, 46, 44, 15, 24], which update Transmission of privatized updates After completing the local update to θ i , we deﬁne the local diﬀerence ∆ i = θ i − θ and transmit this to the aggregator using a separated mechanism (Deﬁnition 2.2).”
Memory and Communication Efficient Distributed Stochastic Optimization with Minibatch Prox (2017, methodology): “, 2006) and has been considered in various contexts (Kulis and Bartlett, 2010; Toulis and Airoldi, 2014).”
Interaction Matters: A Note on Non-asymptotic Local Convergence of Generative Adversarial Networks (2018, background): “it Updates Implicit Update (IU) rules have been shown to be more robust compared to explicit updates, and typically match the performance or even outperform the latter empirically in online learning [Kulis and Bartlett, 2010]. We will show that a simple adaptation of implicit updates for simultaneou”
Efficient mini-batch training for stochastic optimization (2014, background): “For example, [16] proposed to solve minw φIt(w) directly, while [3] presented a L-BFGS style updating.”
Accurate Inference for Adaptive Linear Models (2017, methodology): “ed W-decorrelation decorrelation as an online procedure for optimizing the bias-variance tradeo objective (6), it holds a dual interpretation as implicit stochastic gradient descent (SGD) [see, e.g., Kulis and Bartlett, 2010] (also known as incremental proximal minimization [Bertsekas, 2011] or the ”
Proposal Scaling Distributed Machine Learning with System and Algorithm Co-design (2016, unknown): “For example, [77] proposed to solve minw fIt(w) directly, while [25] presented a L-BFGS style updating.”
Stochastic (Approximate) Proximal Point Methods: Convergence, Optimality, and Adaptivity (2018, unknown): “as [7] analyzes stochastic proximal point algorithms in an incremental framework (when S= f1;:::;mgis a nite set), showing convergence results similar to subgradient methods, while Kulis and Bartlett [30] and Karampatziakis and Langford [28] give theoretical and empirical results in online convex op”
Differentially Private Online Learning (2011, unknown): “The analysis is similar to our analysis for IGD and requires a fairly straightforward modiﬁcation of the regret analysis by [27].”
The Interplay Between Stability and Regret in Online Learning (2012, methodology): “Furthermore, our O(lnT ) proof requires only strong-convexity and Lipschitz continuity, in contrast to strong-convexity and Lipschitz continuity of the gradient in [12].”
Online Importance Weight Aware Updates (2010, background): “Among the other algorithms we consider, implicit updates [14, 15] turn out to coincide with importance invariant ones for piecewise linear losses and provide qualitatively similar updates for other losses, implying that our derivation is an alternative way to motivate this style of algorithm.”
A survey of Algorithms and Analysis for Adaptive Online Learning (2014, background): “update, since for general convex ℓt it is no longer possible to solve for ˆxt+1 in closed form. The implicit update was introduced by Kivinen and Warmuth (1997), and has more recently been studied by Kulis and Bartlett (2010). Again considering the Ψ terms, the composite Mirror Descent update Eq. (1”