Federated Learning with Matched Averaging

By Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, Yasaman Khazaeni. 2020

1,404 citations recorded by the source corpus.

Federated learning allows edge devices to collaboratively learn a shared model while keeping the training data on device, decoupling the ability to do model training from the need to store the data in the cloud. We propose Federated matched averaging (FedMA) algorithm designed for federated learning of modern neural network architectures e.g. convolutional neural networks (CNNs) and LSTMs. FedMA constructs the shared global model in a layer-wise manner by matching and averaging hidden elements (i.e. channels for convolution layers; hidden states for LSTM; neurons for fully connected layers) with similar feature extraction signatures. Our experiments indicate that FedMA outperforms popular state-of-the-art federated learning algorithms on deep CNN and LSTM architectures trained on real world datasets, while improving the communication efficiency.

Subfields: Artificial Intelligence, Building and Construction, Optimization & Training, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Qiang Yang
  2. Mohit Bansal
  3. Dacheng Tao
  4. Yang Liu
  5. Quoc V. Le
  6. Jianfeng Gao
  7. Percy Liang
  8. Yoshua Bengio

In Building and Construction

  1. Bingsheng He
  2. Qinbin Li
  3. Murali Annavaram
  4. Chaoyang He
  5. Guodong Long
  6. Jing Jiang
  7. Qiang Yang
  8. Tianyi Zhou

In Optimization & Training

  1. Qiang Yang
  2. Martin Jaggi
  3. Aurélien Bellet
  4. Peter Richtárik
  5. Ji Liu
  6. Chaoyang He
  7. Sebastian U. Stich
  8. Wei-Lun Chao

In Federated & Distributed Learning

  1. Qiang Yang
  2. Yang Liu
  3. Chaoyang He
  4. Murali Annavaram
  5. Bingsheng He
  6. Qinbin Li
  7. Amir Salman Avestimehr
  8. Tianjian Chen

Citations in the corpus