Personalized Federated Learning with Moreau Envelopes

By Canh T. Dinh, Nguyen H. Tran, Tuan Dung Nguyen. 2020

1,438 citations recorded by the source corpus.

Federated learning (FL) is a decentralized and privacy-preserving machine learning technique in which a group of clients collaborate with a server to learn a global model without sharing clients' data. One challenge associated with FL is statistical diversity among clients, which restricts the global model from delivering good performance on each client's task. To address this, we propose an algorithm for personalized FL (pFedMe) using Moreau envelopes as clients' regularized loss functions, which help decouple personalized model optimization from the global model learning in a bi-level problem stylized for personalized FL. Theoretically, we show that pFedMe's convergence rate is state-of-the-art: achieving quadratic speedup for strongly convex and sublinear speedup of order 2/3 for smooth nonconvex objectives. Experimentally, we verify that pFedMe excels at empirical performance compared with the vanilla FedAvg and Per-FedAvg, a meta-learning based personalized FL algorithm.

Subfields: Artificial Intelligence, Optimization & Training, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Dacheng Tao
  2. Qiang Yang
  3. Percy Liang
  4. Mohit Bansal
  5. Yoshua Bengio
  6. Bo Du
  7. Ruslan Salakhutdinov
  8. Jianfeng Gao

In Optimization & Training

  1. Qiang Yang
  2. Dacheng Tao
  3. Tian Li
  4. H. Vincent Poor
  5. Bingsheng He
  6. Qinbin Li
  7. Virginia Smith
  8. Khaled Ben Letaief

In Federated & Distributed Learning

  1. Qiang Yang
  2. Bingsheng He
  3. Qinbin Li
  4. H. Vincent Poor
  5. Tian Li
  6. Virginia Smith
  7. Han Yu
  8. Walid Saad

Citations in the corpus