LEAF: A Benchmark for Federated Settings

By Sebastian Caldas, Peter Wu, Tian Li, Jakub Konecný, H. Brendan McMahan, Virginia Smith, Ameet Talwalkar. 2018

1,782 citations recorded by the source corpus.

Modern federated networks, such as those comprised of wearable devices, mobile phones, or autonomous vehicles, generate massive amounts of data each day. This wealth of data can help to learn models that can improve the user experience on each device. However, the scale and heterogeneity of federated data presents new challenges in research areas such as federated learning, meta-learning, and multi-task learning. As the machine learning community begins to tackle these challenges, we are at a critical time to ensure that developments made in these areas are grounded with realistic benchmarks. To this end, we propose LEAF, a modular benchmarking framework for learning in federated settings. LEAF includes a suite of open-source federated datasets, a rigorous evaluation framework, and a set of reference implementations, all geared towards capturing the obstacles and intricacies of practical federated environments.

Subfields: Artificial Intelligence, Evaluation & Benchmarks, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Qiang Yang
  2. Dacheng Tao
  3. Percy Liang
  4. Yoshua Bengio
  5. Sergey Levine
  6. Martin Jaggi
  7. Ruslan Salakhutdinov
  8. Quoc V. Le

In Evaluation & Benchmarks

  1. Ameet Talwalkar
  2. Virginia Smith
  3. Peter Kairouz
  4. Zachary B. Charles
  5. Tian Li
  6. Jakub Konecný
  7. Chelsea Finn
  8. Percy Liang

In Federated & Distributed Learning

  1. Bingsheng He
  2. Qinbin Li
  3. Virginia Smith
  4. Qiang Yang
  5. Aryan Mokhtari
  6. Martin Jaggi
  7. Jakub Konecný
  8. Tian Li

Citations in the corpus