Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification

By Tzu-Ming Harry Hsu, Hang Qi, Matthew Brown. 2019

1,651 citations recorded by the source corpus.

Federated Learning enables visual models to be trained in a privacy-preserving way using real-world data from mobile devices. Given their distributed nature, the statistics of the data across these devices is likely to differ significantly. In this work, we look at the effect such non-identical data distributions has on visual classification via Federated Learning. We propose a way to synthesize datasets with a continuous range of identicalness and provide performance measures for the Federated Averaging algorithm. We show that performance degrades as distributions differ more, and propose a mitigation strategy via server momentum. Experiments on CIFAR-10 demonstrate improved classification performance over a range of non-identicalness, with classification accuracy improved from 30.1% to 76.9% in the most skewed settings.

Subfields: Artificial Intelligence, Computer Science Applications, Computer Vision and Pattern Recognition, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Martin Jaggi
  2. Jakub Konecný
  3. Qiang Yang
  4. Peter Richtárik
  5. Sebastian U. Stich
  6. Dacheng Tao
  7. Virginia Smith
  8. Ananda Theertha Suresh

In Computer Science Applications

  1. Zachary B. Charles
  2. H. Brendan McMahan
  3. Bingsheng He
  4. Qinbin Li
  5. Sashank J. Reddi
  6. Sebastian U. Stich
  7. Galen Andrew
  8. Dawn Song

In Computer Vision and Pattern Recognition

  1. Jiashi Feng
  2. Dacheng Tao
  3. Luc Van Gool
  4. Bernt Schiele
  5. Trevor Darrell
  6. Sebastian U. Stich
  7. Jian Liang
  8. Yi Yang

In Federated & Distributed Learning

  1. Jianyu Wang
  2. Gauri Joshi
  3. Bingsheng He
  4. Qinbin Li
  5. Virginia Smith
  6. Zachary B. Charles
  7. Martin Jaggi
  8. Sebastian U. Stich

Citations in the corpus