Multi-Objective Evolutionary Federated Learning

By Hangyu Zhu, Yaochu Jin. 2018

274 citations recorded by the source corpus.

Federated learning is an emerging technique used to prevent the leakage of private information. Unlike centralized learning that needs to collect data from users and store them collectively on a cloud server, federated learning makes it possible to learn a global model while the data are distributed on the users’ devices. However, compared with the traditional centralized approach, the federated setting consumes considerable communication resources of the clients, which is indispensable for updating global models and prevents this technique from being widely used. In this paper, we aim to optimize the structure of the neural network models in federated learning using a multi-objective evolutionary algorithm to simultaneously minimize the communication costs and the global model test errors. A scalable method for encoding network connectivity is adapted to federated learning to enhance the efficiency in evolving deep neural networks. Experimental results on both multilayer perceptrons and convolutional neural networks indicate that the proposed optimization method is able to find optimized neural network models that can not only significantly reduce communication costs but also improve the learning performance of federated learning compared with the standard fully connected neural networks.

Subfields: Artificial Intelligence, Computer Science Applications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Hangyu Zhu
2. Dusist Niyato
3. Zehui Xiong
4. Jiawen Kang
5. Qinbin Li
6. Bingsheng He
7. Zhaomin Wu
8. Zeyi Wen

In Computer Science Applications

1. Shiqing Liu
2. Jinjin Xu
3. Yaochu Jin
4. Hangyu Zhu
5. Jiawen Kang
6. Yang Chen
7. Bingsheng He
8. Azzam Mourad

Citations in the corpus

• Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approach (2019, background): “More details about the basics of federated learning can be found in [4], [2].”
• Real-Time Federated Evolutionary Neural Architecture Search (2020, neutral): “In addition, Zhu et al. [7] uses a multi-objective evolutionary algorithm (MOEA) to simultaneously enhance the model performance and communication efficiency.”
• Communication-Efficient Federated Deep Learning With Layerwise Asynchronous Model Update and Temporally Weighted Aggregation (2019, background): “Notmuch work has been reported on reducing the communication cost by reducing the number of parameters to be uploaded and downloaded between clients and the server except for some recent work reported most recently [18].”
• A review of applications in federated learning (2020, background): “100210 Leroy, D., Coucke, A., Lavril, T., Gisselbrecht, T. and Dureau, J. (2019). Federated Learning for Keyword Spotting. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 6341–6345. https://doi.org/10.1109/ICASSP.2019.8683546 Li, H. and Han, T. (”
• Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges (2022, background): “Which are known to thwart the effectiveness of the attack from Zhu and Jin [166], so we evaluate whether these defences are also applicable to the other attacks.”
• Federated Learning for Healthcare Informatics (2019, background): “sparse, low-rank [50], or more specifically, pruning the least useful connections in a network [29,100], weight quantization [15,81], and model distillation [35].”
• Federated Learning on Non-IID Data: A Survey (2021, methodology): “[168] use a multi-objective evolutionary algorithm to simultaneously increase the model performance and decrease communication costs.”
• Incentive Mechanism for Reliable Federated Learning: A Joint Optimization Approach to Combining Reputation and Contract Theory (2019, background): “The federated learning aims to optimize a global loss function ( ) through minimizing the weighted average of every mobile device n’s local loss function n( ) [20] on its local dataset, i.”
• Artificial intelligence in recommender systems (2020, background): “Due to its capability of handling multiple objectives, new requirements can be taken into account in designing recommender systems, in addition to accuracy and diversity [182].”
• Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems (2021, background): “In [93], the authors seek to minimize the communication cost in federated learning without increasing the global model test”
• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection (2019, background): “How to reasonably set the number of communication rounds is also promising [208].”
• A Learning-Based Incentive Mechanism for Federated Learning (2020, background): “The goal is to optimize the global loss function F(θ) by minimizing the weighted average of every edge node n’s local loss function Fn(θ) on its local training samples [35], i.”
• Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues (2020, background): “Recent study [11] formulates FL as biobjective optimization problem with two objectives of communication cost minimization and learning accuracy maximization.”
• A Survey on federated learning* (2020, background): “Zhu & Jin optimized sparse evolutionary training thus convey only piece of parameters to server [25].”