Federated Meta-Learning with Fast Convergence and Efficient Communication

By Fei Chen, Romy Mi Luo, Zhenhua Dong, Zhenguo Li, Xiuqiang He. 2018

475 citations recorded by the source corpus.

Statistical and systematic challenges in collaboratively training machine learning models across distributed networks of mobile devices have been the bottlenecks in the real-world application of federated learning. In this work, we show that meta-learning is a natural choice to handle these issues, and propose a federated meta-learning framework FedMeta, where a parameterized algorithm (or meta-learner) is shared, instead of a global model in previous approaches. We conduct an extensive empirical evaluation on LEAF datasets and a real-world production dataset, and demonstrate that FedMeta achieves a reduction in required communication cost by 2.82-4.33 times with faster convergence, and an increase in accuracy by 3.23%-14.84% as compared to Federated Averaging (FedAvg) which is a leading optimization algorithm in federated learning. Moreover, FedMeta preserves user privacy since only the parameterized algorithm is transmitted between mobile devices and central servers, and no raw data is collected onto the servers.

Subfields: Artificial Intelligence, Meta & Continual Learning, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Meta & Continual Learning

In Federated & Distributed Learning

Citations in the corpus

Personalized Federated Learning with Moreau Envelopes (2020, neutral): “The application of federated meta-learning in recommender systems was studied in [37].”
Personalized Federated Learning: A Meta-Learning Approach (2020, background): “Moreover, Chen et al. (2018) focused on recommendation systems and proposed a meta-federated learning framework in which a parameterized meta-algorithm is used to train parameterized recommendation models and both meta-algorithm and local models’ parameters need to be optimized.”
K For The Price Of 1: Parameter Efficient Multi-task And Transfer Learning (2018, background): “Our work has interesting connections to meta-learning (Nichol & Schulman, 2018; Finn et al., 2017; Chen et al., 2018).”
Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift (2020, methodology): “Also of particular interest are methods for robustness or adaptation to different users [22, 8, 28, 13, 37], a setting commonly referred to as robust or fair federated learning [43, 46, 34].”
FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare (2019, background): “Since then, other researchers started to focus on privacy-preserving machine learning [Bonawitz et al. , 2017; Shokri and Shmatikov, 2015; Geyer et al. , 2017], federated multi-task learning [Smith et al. , 2017 ] , as well as personalized federated learning [ Chen et al. , 2018a ] .”
FedHome: Cloud-Edge Based Personalized Federated Learning for In-Home Health Monitoring (2020, background): “There are also some research efforts targeting at making federated learning more personalizable [53].”
Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, unknown): “[25] propose a federated meta-learning framework, FedMeta.”
Ditto: Fair and Robust Federated Learning Through Personalization (2020, background): “…Sattler et al., 2020; Muhammad et al., 2020), finetuning/transfer learning (Zhao et al., 2018; Yu et al., 2020), meta-learning (Jiang et al., 2019; Chen et al., 2018; Khodak et al., 2019; Fallah et al., 2020; Li et al., 2020a; Singhal et al., 2021), and other forms of MTL, such as hard model…”