Federated Learning with Autotuned Communication-Efficient Secure Aggregation

By Keith Bonawitz, Fariborz Salehi, Jakub Konecný, H. Brendan McMahan, Marco Gruteser. 2019

84 citations recorded by the source corpus.

Federated Learning enables mobile devices to collaboratively learn a shared inference model while keeping all the training data on a user’s device, decoupling the ability to do machine learning from the need to store the data in the cloud. Existing work on federated learning with limited communication demonstrates how random rotation can enable users’ model updates to be quantized much more efficiently, reducing the communication cost between users and the server. Meanwhile, secure aggregation enables the server to learn an aggregate of at least a threshold number of device’s model contributions without observing any individual device’s contribution in unaggregated form. In this paper, we highlight some of the challenges of setting the parameters for secure aggregation to achieve communication efficiency, especially in the context of the aggressively quantized inputs enabled by random rotation. We then develop a recipe for auto-tuning communication-efficient secure aggregation, based on specific properties of random rotation and secure aggregation – namely, the predictable distribution of vector entries post-rotation and the modular wrapping inherent in secure aggregation. We present both theoretical results and initial experiments.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Federated & Distributed Learning, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Federated & Distributed Learning

In Privacy & Security

Citations in the corpus

Advances and Open Problems in Federated Learning (2019, background): “These challenges could be addressed by optimization algorithms that are robust to hyperparameter settings (the same hyperparameter values work for many different real world datasets and architectures), as well as adaptive or self-tuning algorithms [381, 75].”
The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation (2021, background): “Federated Learning Under FL, a set of clients (e.g., mobile devices or institutions) collaboratively train a model under the orchestration of a central server, while keeping training data decentralized (McMahan et al., 2017; Bonawitz et al., 2019a).”
Byzantine-Resilient Secure Federated Learning (2020, methodology): “More recent approaches have focused on reducing the communication overhead, by training in a smaller parameter space [20], autotuning the parameters [21], or by utilizing coding techniques [6].”
Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning (2020, neutral): “In [32], a framework has been proposed for autotuning the parameters in secure federated learning, to achieve communication-efficiency.”
Differentially Private Learning with Adaptive Clipping (2019, neutral): “Suppose at some round we have n samples of X , with values ( x 1 , . . . , x n ).”
Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges (2022, methodology): “[140] defined an efficient and robust SMPC protocol for the aggregation procedure and, later on, studied its parameter selection [141].”
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection (2019, background): “Thus, efficiency is an important metric in FLSs and many people have been working on reducing the overheads, especially communication size [95, 122, 156, 26].”