Federated Evaluation of On-device Personalization

By Kangkang Wang, Rajiv Mathews, Chloé Kiddon, Hubert Eichner, Franccoise Beaufays, Daniel Ramage. 2019

321 citations recorded by the source corpus.

Federated learning is a distributed, on-device computation framework that enables training global models without exporting sensitive user data to servers. In this work, we describe methods to extend the federation framework to evaluate strategies for personalization of global models. We present tools to analyze the effects of personalization and evaluate conditions under which personalization yields desirable models. We report on our experiments personalizing a language model for a virtual keyboard for smartphones with a population of tens of millions of users. We show that a significant fraction of users benefit from personalization.

Subfields: Artificial Intelligence, Information Systems, Transportation, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Transportation

In Optimization & Training

Citations in the corpus

Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, background): “On-Device Learning On-device AI is becoming more ubiquitously adopted to enable model personalization [82], [83], [84] while improving data privacy [85], [86], [87], yet its impact in terms of carbon emission is often overlooked.”
Advances and Open Problems in Federated Learning (2019, background): “Different clients are likely to use language differently, and in fact on-device personalization of model parameters has yielded significant improvements for this problem [403].”
Three Approaches for Personalization with Applications to Federated Learning (2020, methodology): “One empirically popular approach to overcome this is the ﬁne-tuning approach, where the central model is ﬁne-tuned on the local data (Wang et al., 2019).”
LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasets (2020, background): “Existing works achieve personalization via meta-learning [4–6], multi-task learning [7, 8], transfer learning [9, 10], etc.”
Towards Personalized Federated Learning (2021, background): “As each client experiences a different baseline accuracy due to statistical data heterogeneity, measuring the changes in model accuracy before and after personalization is a useful approach to assess the benefits of personalization [30, 87, 89].”
FedBABU: Towards Enhanced Representation for Federated Image Classification (2021, methodology): “We first investigate personalization of a single global model using the FedAvg algorithm, following [42, 23], to connect a single global model to multiple personalized models.”
Personalized Federated Learning: An Attentive Collaboration Approach (2020, methodology): “The most practical way to build a personalized model for a client is to customize a global model by local ﬁne-tuning [10, 11, 12, 20, 21, 22], where the parameters of the global model are used as the initialization to train a personalized model on the local data of every client.”
Personalized Cross-Silo Federated Learning on Non-IID Data (2020, methodology): “FedAvg-FT and FedProx-FT achieve good performance by taking many ﬁne-tuning steps to tune the poor global models back to normal.”
Federated Learning on Non-IID Data: A Survey (2021, methodology): “In [132], the authors train a next-word prediction model using FedAvg with different hyperparameters for personalization to make sure the model can generalize in new mobile devices.”
Personalized Federated Learning with First Order Model Optimization (2020, background): “…Learning with Non-IID Data While ﬁne-tuning a global model on a client’s local data is a natural strategy to personalize (Mansour et al., 2020; Wang et al., 2019), prior work has shown that non-IID decentralized data can introduce challenges such as parameter divergence (Zhao et al., 2018),…”
Ditto: Fair and Robust Federated Learning Through Personalization (2020, background): “The benefits of personalization for fairness alone have been demonstrated empirically in prior work (Wang et al., 2019; Hao et al., 2020).”
Survey of Personalization Techniques for Federated Learning (2020, methodology): “Transfer learning has also been used in federated settings, e.g. Wang et al. [20], where some or all parameters of a trained global model are re-learned on lo-cal data.”