Distributed Fixed Point Methods with Compressed Iterates
By S. Chraibi, Ahmed Khaled, Dmitry Kovalev, Peter Richtárik, Adil Salim, Martin Tak'avc. 2019
18 citations recorded by the source corpus.
We propose basic and natural assumptions under which iterative optimization methods with compressed iterates can be analyzed. This problem is motivated by the practice of federated learning, where a large model stored in the cloud is compressed before it is sent to a mobile device, which then proceeds with training based on local data. We develop standard and variance reduced methods, and establish communication complexity bounds. Our algorithms are the first distributed methods with compressed iterates, and the first fixed point methods with compressed iterates.
Subfields: Artificial Intelligence, Computational Mechanics, Numerical Analysis, Efficiency & Compression, Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Dmitry Kovalev
- Peter Richtárik
- Laurent Condat
- Grigory Malinovsky
- Elnur Gasanov
- Nicolas Loizou
- Filip Hanzely
- Jakub Konecný
In Computational Mechanics
- Dmitry Kovalev
- Peter Richtárik
- Grigory Malinovsky
- Laurent Condat
- Elnur Gasanov
- Nicolas Loizou
- Martin Takác
- Jakub Konecný
In Numerical Analysis
- Peter Richtárik
- Grigory Malinovsky
- Dmitry Kovalev
- Elnur Gasanov
- Laurent Condat
- Nicolas Loizou
- Wotao Yin
- Ming Yan
In Efficiency & Compression
In Optimization & Training
- Dmitry Kovalev
- Peter Richtárik
- Laurent Condat
- Grigory Malinovsky
- Elnur Gasanov
- Nicolas Loizou
- Filip Hanzely
- Jakub Konecný
Citations in the corpus
- From Local SGD to Local Fixed Point Methods for Federated Learning (2020, methodology): “We will also investigate compression [10, 11] of the communicated variables, with or without variance reduction, in combination with locality.”