Ditto: Fair and Robust Federated Learning Through Personalization

By Tian Li, Shengyuan Hu, Ahmad Beirami, Virginia Smith. International Conference on Machine Learning, 2020

1,411 citations recorded by the source corpus.

Fairness and robustness are two important concerns for federated learning systems. In this work, we identify that robustness to data and model poisoning attacks and fairness, measured as the uniformity of performance across devices, are competing constraints in statistically heterogeneous networks. To address these constraints, we propose employing a simple, general framework for personalized federated learning, Ditto, that can inherently provide fairness and robustness benefits, and develop a scalable solver for it. Theoretically, we analyze the ability of Ditto to achieve fairness and robustness simultaneously on a class of linear problems. Empirically, across a suite of federated datasets, we show that Ditto not only achieves competitive performance relative to recent personalization methods, but also enables more accurate, robust, and fair models relative to state-of-the-art fair or robust baselines.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/31949039a48961f939ac50440c3b4b8504fccceb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qiang Yang
2. Han Yu
3. Bo Du
4. Dacheng Tao
5. Guodong Long
6. Jing Jiang
7. Mang Ye
8. Tianyi Zhou

Citations in the corpus

• Federated Multi-Task Learning under a Mixture of Distributions (2021, background): “Other MTL-based approaches [24, 23, 16, 26, 36] are able to train more general models at the cost of considering simpler penalization terms (e.”
• Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, methodology): “Ditto [122] is a scalable FMTL framework with two tasks, a global objective and a local objective.”
• Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learning (2021, background): “Personalization Ditto [37] EWC [71] 86.”
• Towards Personalized Federated Learning (2021, background): “As such, there are PFL works that use distribution-based evaluation frameworks such as histogram profiling [61, 87], variance metrics [37, 55, 88] and metrics at the individual client level [24, 43] to evaluate the performance of personalized models.”
• FedALA: Adaptive Local Aggregation for Personalized Federated Learning (2022, methodology): “Tran, and Nguyen 2020) and Ditto (Li et al. 2021a), and (3) methods that learn local models with personalized (local) aggregation, including FedAMP (Huang et al.”
• Federated Learning from Pre-Trained Models: A Contrastive Learning Approach (2022, background): “[29] Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith.”
• A Metaverse: Taxonomy, Components, Applications, and Open Challenges (2022, methodology): “[334] proposed to use Ditto, a simple and general framework for”
• FedBABU: Towards Enhanced Representation for Federated Image Classification (2021, methodology): “In summary, FedAvg are comparable and better than even recent personalized FL algorithms such as Per-FedAvg and Ditto, and FedBABU (ours) achieves the higher performance than FedAvg and other decoupling algorithms.”