Noname manuscript No. (will be inserted by the editor) Incremental Proximal Methods for Large Scale Convex Optimization

Citations in the corpus

• Protection Against Reconstruction and Its Applications in Private Federated Learning (2018, methodology): “An alternative is to use stochastic proximal-point-type updates [10, 46, 44, 15, 24], which update Transmission of privatized updates After completing the local update to θ i , we define the local difference ∆ i = θ i − θ and transmit this to the aggregator using a separated mechanism (Definition 2.2).”
• Communication-efficient algorithms for decentralized and stochastic optimization (2017, positive): “recent years due to the emergence of large-scale networks. The seminal work on distributed optimization [60,59] has been followed by distributed incremental (sub)gradient methods and proximal methods [36,48,2,61], and more recently the incremental aggregated gradient methods and its proximal variant”
• A Proximal Stochastic Gradient Method with Progressive Variance Reduction (2014, methodology): “The Prox-SG method can be viewed as a variant of the randomized incremental proximal algorithms proposed in [3].”
• Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs (2016, methodology): “Some earlier methods include distributed incremental (sub)gradient methods [37–39, 54 ] and incremental incremental proximal methods [ 3 , 68] , while a more recent Some distributed methods exist that do not (explicitly) use dual variables but can still converge to an exact consensual solution…”
• Memory and Communication Efficient Distributed Stochastic Optimization with Minibatch Prox (2017, methodology): “s functions were analyzed in the online learning setting (Cheng et al., 2006; Kulis and Bartlett, 2010). For finite-sum objectives, methods basedonincremental/stochastic proximal updates werestudied inBertsekas (2011);Defazio(2016). Needell and Tropp (2014) analyzed a randomized block Kaczmarz method”
• The importance of better models in stochastic optimization (2019, methodology): “In the convex setting (8, 20, 21), the stochastic proximal point method uses the model fx(y; s) := f(y; s); in the weakly convex setting, we regularize and use”
• Accurate Inference for Adaptive Linear Models (2017, background): “, Kulis & Bartlett, 2010), also known as incremental proximal minimization (Bertsekas, 2011) or the normalized least mean squares filter (Nagumo & Noda, 1967) in this context, with step-size λ applied to the least-squares objective, 1 n ∑n i=1(yi − 〈β,xi〉)(2).”
• Stochastic (Approximate) Proximal Point Methods: Convergence, Optimality, and Adaptivity (2018, methodology): “Any aProx method (4) recovers standard convergence guarantees of stochastic gradient and proximal point methods [42, 7] (see [14] for the basic convergence guarantee).”
• SMART: The Stochastic Monotone Aggregated Root-Finding Algorithm (2016, background): “These two structural assumptions, and the host of algorithms that adopt them, have led to big improvements in large-scale algorithm design (for examples, see [28,17,34,36,35,16,20,15,25,29,9,5,8,37,4, 27,6,31]).”
• A Decentralized Proximal-Gradient Method With Network Independent Step-Sizes and Separated Convergence Rates (2017, background): “The incremental algorithms [3,17–19,31,42] can solve (1) without the need of a “master” agent but requires that the network contains a ring structure.”
• How catastrophic can catastrophic forgetting be in linear regression? (2022, unknown): “The stochastic proximal point algorithm (SPPA) [8, 54, 60] follows an update rule (for a step size η) of wt=argminw ( 1 2(Xτ(t)w−yτ(t)) 2 + 12η ‖w−wt−1‖ 2 ) , equivalent when η→∞ to our rule in Eq.”