Convergence Analysis of Inexact Randomized Iterative Methods

By Nicolas Loizou, Peter Richtárik. 2019

23 citations recorded by the source corpus.

The first author would like to acknowledge Robert Mansel Gower, Georgios Loizou and Rachael Tappenden for useful discussions.

Subfields: Artificial Intelligence, Computational Mechanics, Numerical Analysis, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Nicolas Loizou
2. Peter Richtárik
3. Simon Lacoste-Julien
4. Robert Mansel Gower
5. Ioannis Mitliagkas
6. Gauthier Gidel
7. Hugo Berard
8. Issam Hadj Laradji

In Computational Mechanics

1. Nicolas Loizou
2. Peter Richtárik
3. Simon Lacoste-Julien
4. Ioannis Mitliagkas
5. Gauthier Gidel
6. Robert Mansel Gower
7. Hugo Berard
8. Issam Hadj Laradji

In Numerical Analysis

1. Nicolas Loizou
2. Peter Richtárik
3. Simon Lacoste-Julien
4. Robert Mansel Gower
5. Issam Hadj Laradji
6. Sharan Vaswani
7. Othmane Sebbouh
8. Gauthier Gidel

In Optimization & Training

1. Nicolas Loizou
2. Peter Richtárik
3. Simon Lacoste-Julien
4. Ioannis Mitliagkas
5. Robert Mansel Gower
6. Gauthier Gidel
7. Hugo Berard
8. Othmane Sebbouh

Citations in the corpus

• Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivity (2021, methodology): “…might be restrictive or never be satisfied 5 and proposed alternative conditions [Nguyen et al., 2018, Vaswani et al., 2018, Gower et al., 2019, 2021, Khaled et al., 2020, Khaled and Richtárik, 2020, Assran et al., 2019, Koloskova et al., 2020, Patel and Zhang, 2021, Loizou et al., 2020, 2021].”
• Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence (2020, methodology): “[29] Nicolas Loizou and Peter Richtárik.”
• Momentum and stochastic momentum for stochastic gradient, Newton, proximal point and subspace descent methods (2017, background): “This result sparked renewed interest in design of randomized methods for solving linear systems [19,49,51,53,55,57,58,81,97].”
• SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation (2020, methodology): “[29] Nicolas Loizou and Peter Richtárik.”
• Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsize (2021, background): “…the training problem of an over-parametrized model such as a deep neural network (Ma et al., 2018) or involves solving a consistent linear system (Loizou & Richtárik, 2020b;a) or problems such as deep matrix factorization (Rolinek & Martius, 2018; Vaswani et al., 2019b), each individual loss…”
• Revisiting Randomized Gossip Algorithms: General Framework, Convergence Rates and Novel Block and Accelerated Protocols (2019, background): “Inexact variants of Algorithm 1 have been proposed in [50].”