A Superquantile Approach to Federated Learning with Heterogeneous Devices

By Yassine Laguel, Krishna Pillutla, Jérôme Malick, Zaïd Harchaoui. 2021

28 citations recorded by the source corpus.

We present a federated learning framework that allows one to handle heterogeneous client devices that may not conform to the population data distribution. The proposed approach hinges upon a parameterized superquantile-based objective, where the parameter ranges over levels of conformity. We introduce a stochastic optimization algorithm compatible with secure aggregation, which interleaves device filtering steps with federated averaging steps. We conclude with numerical experiments with neural networks on computer vision and natural language processing data.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

Robust Aggregation for Federated Learning (2019, background): “Extensions were proposed in [7, 29, 33, 47, 50, 57, 69, 75, 79]; see also the recent surveys [46, 56].”
Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, methodology): “[107, 108, 170] improves the performance of the worse-off clients while maintaining the average performance using a risk measure known as the superquantile (or CVaR), which is able to capture the tail statistical characteristics of the client error distribution.”
Federated Learning in Edge Computing: A Systematic Survey (2022, background): “Furthermore, the authors of [176] presented a federated learning framework that allows one to handle heterogeneous client devices that couples a parameterized superquantile-based objective.”
On Large-Cohort Training for Federated Learning (2021, background): “Such work has focused on various aspects, including communicationefficiency [5, 34, 37, 52], data and systems heterogeneity [25, 28, 29, 39–42, 67], and fairness [26, 43].”
Tilted Empirical Risk Minimization (2020, background): “As discussed in Section 2, the TERM objective can be viewed as a minimax smoothing (Kort & Bertsekas, 1972; Pee & Royset, 2011) with the added flexibility of a tunable t to allow the user to optimize utility for different quantiles of loss similar to superquantile approaches (Laguel et al., 2021; Ro”