FedBCD: A Communication-Efficient Collaborative Learning Framework for Distributed Features

By Yang Liu, Yan Kang, Xinwei Zhang, Liping Li, Yong Cheng, Tianjian Chen, Mingyi Hong, Qiang Yang. 2022

139 citations recorded by the source corpus.

We introduce a novel federated learning framework allowing multiple parties having different sets of attributes about the same user to jointly build models without exposing their raw data or model parameters. Conventional federated learning approaches are inefficient for cross-silo problems because they require the exchange of messages for gradient updates at every iteration, and raise security concerns over sharing such messages during learning. We propose a Federated Stochastic Block Coordinate Descent (FedBCD) algorithm, allowing each party to conduct multiple local updates before each communication to effectively reduce communication overhead. Under a practical security model, we show that parties cannot infer others' exact raw data (“deep leakage”) from collections of messages exchanged in our framework, regardless of the number of communication to be performed. Further, we provide convergence guarantees and empirical evaluations on a variety of tasks and datasets, demonstrating significant improvement inefficiency.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yang Liu
2. Han Yu
3. Qiang Yang
4. Tianjian Chen
5. Yan Kang
6. Siwei Feng
7. Qian Xu
8. Tao Fan

Citations in the corpus

• Backdoor attacks and defenses in feature-partitioned collaborative learning (2020, background): “However, featurepartitioned collaborative learning (Yang et al., 2019a; Hu et al., 2019b; Liu et al., 2019) can be another important scenario in many real world applications.”
• FATE: An Industrial Grade Platform for Collaborative Learning With Data Protection (2021, background): “In (Liu et al., 2019) efficiency is improved by introducing multiple local update strategies.”
• Vertical Federated Learning: Concepts, Advances, and Challenges (2022, methodology): “[18] proposed a federated stochastic block coordinate descent algorithm, called FedBCD, that allows each party to conduct multiple client updates before each communication to reduce the number of synchronizations, thereby mitigating the communication overhead.”
• Multi-Participant Multi-Class Vertical Federated Learning (2020, methodology): “Techniques such as those reported in [Liu et al., 2019] can be used to improve the communication efficiency.”