Fair Resource Allocation in Federated Learning
By Tian Li, Maziar Sanjabi, Virginia Smith. 2019
1,024 citations recorded by the source corpus.
Federated learning involves training statistical models in massive, heterogeneous networks. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the devices. In this work, we propose q-Fair Federated Learning (q-FFL), a novel optimization objective inspired by fair resource allocation in wireless networks that encourages a more fair (specifically, a more uniform) accuracy distribution across devices in federated networks. To solve q-FFL, we devise a communication-efficient method, q-FedAvg, that is suited to federated networks. We validate both the effectiveness of q-FFL and the efficiency of q-FedAvg on a suite of federated datasets with both convex and non-convex models, and show that q-FFL (along with q-FedAvg) outperforms existing baselines in terms of the resulting fairness, flexibility, and efficiency.
Subfields: Artificial Intelligence, Computer Science Applications, Federated & Distributed Learning, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Peter Richtárik
- Jakub Konecný
- Qiang Yang
- Aryan Mokhtari
- Virginia Smith
- Martin Jaggi
- Jianyu Wang
- Gauri Joshi
In Computer Science Applications
- Gauri Joshi
- Jianyu Wang
- Zachary B. Charles
- Jakub Konecný
- Carole-Jean Wu
- Young Geun Kim
- Yae Jee Cho
- Dusist Niyato
In Federated & Distributed Learning
- Virginia Smith
- Jianyu Wang
- Gauri Joshi
- Jakub Konecný
- Aryan Mokhtari
- Zachary B. Charles
- Tian Li
- Mehrdad Mahdavi
In AI Ethics, Fairness & Society
- Gauri Joshi
- Jianyu Wang
- Zachary B. Charles
- Jakub Konecný
- Carole-Jean Wu
- Young Geun Kim
- Yae Jee Cho
- Dusist Niyato
Citations in the corpus
- Flower: A Friendly Federated Learning Research Framework (2020, methodology): “Finally, there is an increasing emphasis on the performance of federated optimization in heterogeneous data and system settings (Smith et al., 2017; Li et al., 2018; 2019).”
- Byzantine-Resilient Secure Federated Learning (2020, background): “Another important direction in federated learning is the study of fairness and how to avoid biasing the model towards specific users [28], [29].”
- Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization (2020, methodology): “Moreover, FedNova is also compatible with and complementary to gradient compression/quantization [43–48] and fair aggregation techniques [49, 50].”
- On the Outsized Importance of Learning Rates in Local Update Methods (2020, methodology): “…of federated learning with topics including differential privacy (McMahan et al., 2018; Augenstein et al., 2020), fairness (Mohri et al., 2019; Li et al., 2020b), robustness (Ghosh et al., 2019; Bagdasaryan et al., 2018; Sun et al., 2019), and communication-efficiency (Konečný et al., 2016;…”
- Personalized Federated Learning: A Meta-Learning Approach (2020, background): “…paper, there are several works that study statistical heterogeneity of users’ data points in FL (Zhao et al., 2018; Sahu et al., 2018; Karimireddy et al., 2019; Haddadpour and Mahdavi, 2019; Khaled et al., 2019; Li et al., 2019), but they do not attempt to find a personalized solution for each user.”
- LEAF: A Benchmark for Federated Settings (2018, background): “Our synthetic dataset (introduced in Section 2) is inspired by the one presented in [20], but has possible additional heterogeneity designed to make current meta-learning methods (such as Reptile [27]) fail.”
- Federated Learning: Challenges, Methods, and Future Directions (2019, background): “Recent works have proposed modified modeling approaches that aim to reduce the variance of the model performance across devices [19], [26], [33].”
- Federated Meta-Learning with Fast Convergence and Efficient Communication (2018, methodology): “Following recent work [11], we study the fairness of FedAvg and FedMeta framework by comparing the final accuracy distribution averaged over multiple runs.”
- WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, neutral): “…binding maps.10 Prospective evaluation of top-performing models highlighted a significant drop in prediction performance across cellular contexts relative to cross-validation performance within training cell types (Keilwagen et al., 2019; Quang and Xie, 2019; Li et al., 2019; Li and Guan, 2019).”
- On the Convergence of Local Descent Methods in Federated Learning (2019, background): “Another recent work [19] suggests fair algorithm for federated learning and evaluates their algorithms empirically.”
- On Large-Cohort Training for Federated Learning (2021, background): “One critical issue in FL is fairness across clients, as minimizing (1) may disadvantage some clients (Mohri et al., 2019; Li et al., 2020c).”
- Federated Adversarial Debiasing for Fair and Transferable Representations (2021, methodology): “The fairness in federated learning was first discussed in [22] where users are thought to have different capability for computation.”
- Federated User Representation Learning (2019, background): “Personalization in FL poses many challenges due to its distributed nature, high communication costs, and privacy constraints [17, 5, 6, 19, 18, 20, 32, 12].”
- A Superquantile Approach to Federated Learning with Heterogeneous Devices (2021, methodology): “We also compare ∆ - FL with other heterogeneity-sensitive objectives, namely q -FFL [8], AFL [6].”