Federated Learning via Over-the-Air Computation

By Kai Yang, Tao Jiang, Yuanming Shi, Zhiguo Ding. 2018

1,082 citations recorded by the source corpus.

The stringent requirements for low-latency and privacy of the emerging high-stake applications with intelligent devices such as drones and smart vehicles make the cloud computing inapplicable in these scenarios. Instead, edge machine learning becomes increasingly attractive for performing training and inference directly at network edges without sending data to a centralized data center. This stimulates a nascent field termed as federated learning for training a machine learning model on computation, storage, energy and bandwidth limited mobile devices in a distributed manner. To preserve data privacy and address the issues of unbalanced and non-IID data points across different devices, the federated averaging algorithm has been proposed for global model aggregation by computing the weighted average of locally updated model at each selected device. However, the limited communication bandwidth becomes the main bottleneck for aggregating the locally computed updates. We thus propose a novel over-the-air computation based approach for fast global model aggregation via exploring the superposition property of a wireless multiple-access channel. This is achieved by joint device selection and beamforming design, which is modeled as a sparse and low-rank optimization problem to support efficient algorithms design. To achieve this goal, we provide a difference-of-convex-functions (DC) representation for the sparse and low-rank function to enhance sparsity and accurately detect the fixed-rank constraint in the procedure of device selection. A DC algorithm is further developed to solve the resulting DC program with global convergence guarantees. The algorithmic advantages and admirable performance of the proposed methodologies are demonstrated through extensive numerical results.

Subfields: Computational Mechanics, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Electrical and Electronic Engineering

Citations in the corpus

Age-Based Scheduling Policy for Federated Learning in Mobile Edge Networks (2019, methodology): “…studies have been carried out and resulted in various scheduling protocols for FL, ranging from minimizing the transmission latency [13], maximizing the spectral utility [14], to opportunistically alternating between selecting the UEs with advantageous and disadvantageous channel conditions [15].”
Scheduling Policies for Federated Learning in Wireless Networks (2019, methodology): “When spectral resources become the communication bottleneck, there are new methods exploiting the computeover-air mechanism and a jointly decode-and-average scheme at the edge computing unit [17], [18].”
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications (2020, background): “, global model Θ = (1/N) N n=1 θn with N workers, rather than the individual updates θn, several recent works have utilized uncoded transmissions, so as to harness interference without separate channel allocation [62], [109]–[114].”
Federated Learning for 6G: Applications, Challenges, and Opportunities (2021, positive): “To overcome this challenge, an emerging approach is over-the-air computation, which can gather wireless data quickly by using the superposition nature of wireless transmission [34,35].”
Communication-Efficient Edge AI: Algorithms and Systems (2020, methodology): “The connection between data aggregation from distributed nodes in edge training and the in-network computation problem [20] in wireless sensor networks has been established in [21], which proposed an over-the-air computation approach for fast model”
A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression (2019, background): “A line of works [4, 5, 6] focus on the horizontal structure, in which each node has a subset of data instances with complete data attributes.”
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges (2021, methodology): “As we discussed previously, researchers have also designed AirComp FEEL systems over multiple-antenna channels [38], [112], [113].”
Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues (2020, background): “[57] propose an over-the-air computation-based aggregation approach for FL in wireless environments.”
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications (2021, methodology): “Section II-B: Wireless Techniques for Edge Training Over-the-Air Computation Low-latency analog model aggregation [31], [44], [45]”
One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis (2020, methodology): “Existing works on over-the-air aggregation, as well as its application in the context of FEEL [6], [7], [17]–[19] consider analog modulation assuming that the transmitter can modulate the carrier waveform as desired, freely choosing the I/Q coefficients as arbitrary real number.”
Federated Learning Over Wireless Fading Channels (2019, methodology): “Then we studied the alternative analog transmission approach, cosidered recently in [25]–[27], which does not employ quantization or channel coding, and exploits the superposition property of the wireless MAC, rather than orthogonalizing the transmissions from different devices.”
Broadband Analog Aggregation for Low-Latency Federated Edge Learning (2018, methodology): “A device-selection algorithm is developed in [31] to maximize the number of scheduled devices under a update-distortion constraint.”
Holistic Network Virtualization and Pervasive Network Intelligence for 6G (2023, methodology): “Over-the-air computation based approaches are investigated in [194], [216], [217].”
Over-the-Air Machine Learning at the Wireless Edge (2019, background): “A SIMO wireless MAC with beamforming is considered in [11] for distributed learning, where the goal is to maximize the number of workers scheduled for transmission with an acceptable quality for the retrieved signal.”
Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective (2020, background): “Recognizing the unique physical property of wireless transmission, [2], [20]–[22] propose analog model aggregation over the air, provided that a very stringent synchronization is available.”
Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning (2020, methodology): “ting FL in wireless networks has received many research efforts. To reduce the uploading latency introduced by global model aggregation, novel analog aggregation techniques have been proposed in [15]–[17]. For analog aggregation, the scheduled devices concurrently transmit their local models via ana”