FedMD: Heterogenous Federated Learning via Model Distillation

By Daliang Li, Junpu Wang. 2019

1,187 citations recorded by the source corpus.

Federated learning enables the creation of a powerful centralized model without compromising data privacy of multiple participants. While successful, it does not incorporate the case where each participant independently designs its own model. Due to intellectual property concerns and heterogeneous nature of tasks and data, this is a widespread requirement in applications of federated learning to areas such as health care and AI as a service. In this work, we use transfer learning and knowledge distillation to develop a universal framework that enables federated learning when each agent owns not only their private data, but also uniquely designed models. We test our framework on the MNIST/FEMNIST dataset and the CIFAR10/CIFAR100 dataset and observe fast improvement across all participating models. With 10 distinct participants, the final test accuracy of each model on average receives a 20% gain on top of what's possible without collaboration and is only a few percent lower than the performance each model would have obtained if all private datasets were pooled and made directly available for all participants.

Subfields: Artificial Intelligence, Computer Science Applications, Sociology and Political Science, Efficiency & Compression, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Qiang Yang
  2. Dacheng Tao
  3. Philip S. Yu
  4. Murali Annavaram
  5. Chaoyang He
  6. Lichao Sun
  7. Yang Liu
  8. Guodong Long

In Computer Science Applications

  1. Shiqing Liu
  2. Jinjin Xu
  3. Yaochu Jin
  4. Zachary B. Charles
  5. Hong-You Chen
  6. Wei-Lun Chao
  7. Hangyu Zhu
  8. Jiayu Zhou

In Sociology and Political Science

  1. Lingjuan Lyu
  2. Lichao Sun
  3. Li-zhen Cui
  4. Jun Zhao
  5. Hong-You Chen
  6. Dacheng Tao
  7. Philip S. Yu
  8. Peilin Zhao

In Efficiency & Compression

  1. Murali Annavaram
  2. S. Avestimehr
  3. Chaoyang He
  4. Dacheng Tao
  5. Li-zhen Cui
  6. Hangyu Zhu
  7. Hong-You Chen
  8. Zachary B. Charles

In Federated & Distributed Learning

  1. Qiang Yang
  2. Murali Annavaram
  3. Chaoyang He
  4. Yang Liu
  5. S. Avestimehr
  6. Han Yu
  7. Praneeth Vepakomma
  8. Jiayu Zhou

Citations in the corpus