Real-Time Federated Evolutionary Neural Architecture Search

By Hangyu Zhu, Yaochu Jin. 2020

89 citations recorded by the source corpus.

Federated learning is a distributed machine learning approach to privacy preservation and two major technical challenges prevent a wider application of federated learning. One is that federated learning raises high demands on communication resources, since a large number of model parameters must be transmitted between the server and clients. The other challenge is that training large machine learning models such as deep neural networks in federated learning requires a large amount of computational resources, which may be unrealistic for edge devices such as mobile phones. The problem becomes worse when deep neural architecture search (NAS) is to be carried out in federated learning. To address the above challenges, we propose an evolutionary approach to real-time federated NAS that not only optimizes the model performance but also reduces the local payload. During the search, a double-sampling technique is introduced, in which for each individual, only a randomly sampled submodel is transmitted to a number of randomly sampled clients for training. This way, we effectively reduce computational and communication costs required for evolutionary optimization, making the proposed framework well suitable for real-time federated NAS.

Subfields: Artificial Intelligence, Computer Science Applications, Neural Architecture Search & AutoML, Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yang Liu
2. Qiang Yang
3. Tianjian Chen
4. Xinle Liang
5. Ishika Singh
6. Kunlin Yang
7. Mengxiao Ding
8. Bill Lin

In Computer Science Applications

1. Qiang Yang
2. Yang Liu

In Neural Architecture Search & AutoML

1. Qiang Yang
2. Yang Liu

In Federated & Distributed Learning

1. Yang Liu
2. Qiang Yang
3. Tianjian Chen
4. Jiahuan Luo
5. Yuanqin He
6. Xinle Liang
7. Bingsheng He
8. Qinbin Li

Citations in the corpus

• Self-supervised Cross-silo Federated Neural Architecture Search (2021, methodology): “Zhu and Jin [21] applied NAGA-II [45] to the multiobjective problem in federated NAS tasks for simultaneously optimizing the model performance and local client’s payload.”
• Differentially-private Federated Neural Architecture Search (2020, background): “In (Zhu & Jin, 2020), a federated algorithm is proposed to search neural architectures based on the evolutionary algorithm (EA), which is computationally heavy.”