DJAM: Distributed Jacobi Asynchronous Method for Learning Personal Models

By Inês Almeida, Joao Xavier. 2018

12 citations recorded by the source corpus.

Processing data collected by a network of agents often boils down to solving an optimization problem. The distributed nature of these problems calls for methods that are, themselves, distributed. While most collaborative learning problems require agents to reach a common (or consensus) model, there are situations in which the consensus solution may not be optimal. For instance, agents may want to reach a compromise between agreeing with their neighbors and minimizing a personal loss function. We present DJAM, a Jacobi-like distributed algorithm for learning personalized models. This method is implementation-friendly: it has no hyperparameters that need tuning, it is asynchronous, and its updates only require single-neighbor interactions. We prove that DJAM converges with probability one to the solution, provided that the personal loss functions are strongly convex and have Lipschitz gradient. We then give evidence that DJAM is on par with state-of-the-art methods: our method reaches a solution with error similar to the error of a carefully tuned alternating direction method of multipliers (ADMM) in about the same number of single-neighbor interactions.

Subfields: Artificial Intelligence, Management Science and Operations Research, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Martin Jaggi
2. Qiang Yang
3. Jakub Konecný
4. Peter Richtárik
5. Peter Kairouz
6. Ananda Theertha Suresh
7. Jianyu Wang
8. Gauri Joshi

In Management Science and Operations Research

1. Jakub Konecný
2. Aurélien Bellet
3. Peter Kairouz
4. Zheng Xu
5. Valentina Zantedeschi
6. Marc Tommasi
7. Ananda Theertha Suresh
8. Ramesh Raskar

In Federated & Distributed Learning

1. Qiang Yang
2. Jianyu Wang
3. Gauri Joshi
4. Martin Jaggi
5. Jakub Konecný
6. Mehrdad Mahdavi
7. Yang Liu
8. Chaoyang He

Citations in the corpus

• Advances and Open Problems in Federated Learning (2019, background): “We note that personalization has also been studied in the fully decentralized setting [392, 54, 431, 22], where training individual models is particularly natural.”
• Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs (2019, methodology): “Our work is more closely inspired by recent decentralized approaches that have shown the benefits of collaboratively learning personalized models for each user by leveraging a similarity graph given as input to the algorithm (Vanhaesebrouck et al., 2017; Li et al., 2017; Bellet et al., 2018; Almeida”
• Adaptive Personalized Federated Learning (2020, background): “(2017); Almeida and Xavier (2018), where in addition to the optimization error, they have network constraints or peer-to-peer communication limitation (Bellet et al.”