A Communication Efficient Vertical Federated Learning Framework

By Yang Liu, Yan Kang, Xinwei Zhang, Liping Li, Yong Cheng, Tianjian Chen, Mingyi Hong, Qiang Yang. 2019

79 citations recorded by the source corpus.

One critical challenge for applying today's Artificial Intelligence (AI) technologies to real-world applications is the common existence of data silos across different organizations. Due to legal, privacy and other practical constraints, data from different organizations cannot be easily integrated. Federated learning (FL), especially the vertical FL (VFL), allows multiple parties having different sets of attributes about the same user collaboratively build models while preserving user privacy. However, communication overhead is a principal bottleneck since the existing VFL protocols require per-iteration communications among all parties. In this paper, we propose the Federated Stochastic Block Coordinate Descent (FedBCD) to effectively reduce the communication rounds for VFL. We show that when the batch size, sample size, and the local iterations are selected appropriately, the algorithm requires $\mathcal{O}(\sqrt{T})$ communication rounds to achieve $\mathcal{O}(1/\sqrt{T})$ accuracy. Finally, we demonstrate the performance of FedBCD on several models and datasets, and on a large-scale industrial platform for VFL.

Subfields: Artificial Intelligence, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qiang Yang
2. Yang Liu
3. Bingsheng He
4. Qinbin Li
5. Jiankun Hu
6. Yanming Zhu
7. Xuefei Yin
8. Zeyi Wen

In Federated & Distributed Learning

1. Yang Liu
2. Qiang Yang
3. Bingsheng He
4. Qinbin Li
5. Tianjian Chen
6. Zeyi Wen
7. Praneeth Vepakomma
8. Vivek Sharma

Citations in the corpus

• SplitNN-driven Vertical Partitioning (2020, background): “Vertical federated learning Learning with vertically partitioned data in the context of federated learning, a popular distributed deep learning paradigm was studied in (Nock et al. , 2018; Liu et al. , 2019, 2020; Sameer et al. , 2018; Vepakomma et al. , 2018b; Sharma et al. , 2019a,b).”
• SecureBoost: A Lossless Federated Learning Framework (2019, background): “The concept of vertical FL is first proposed in [5], [11] and protocols are proposed for linear models [5], [13] and neural networks [27].”
• Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems (2021, background): “The authors of [96] address the communication over-”
• Federated Learning in Smart City Sensing: Challenges and Opportunities (2020, unknown): “Where people are recruited into sensing campaigns and are compensated for the data collected by their personal devices [21,22].”
• A Comprehensive Survey of Privacy-preserving Federated Learning (2021, background): “[112] proposed a vertical FL framework based on a block coordinate gradient descent algorithm, in which each client locally conducts more than one gradient updates before sending the local model information to the other clients.”
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey (2019, background): “, more passes over dataset before communication [23], [98] (b) Two-stream training with global model as a reference [99] and (c) Intermediate edge server aggregation [100].”
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection (2019, methodology): “If the parties have the same sample space but different features, one can use FedBCD [111] for NNs and SecureBoost [36] for trees.”