Federated Semi-Supervised Learning with Inter-Client Consistency
By Wonyong Jeong, Jaehong Yoon, Eunho Yang, Sung Ju Hwang. 2020
284 citations recorded by the source corpus.
While existing federated learning approaches mostly require that clients have fully-labeled data to train on, in realistic settings, data obtained at the client side often comes without any accompanying labels. Such deficiency of labels may result from either high labeling cost, or difficulty of annotation due to requirement of expert knowledge. Thus the private data at each client may be only partly labeled, or completely unlabeled with labeled data being available only at the server, which leads us to a new problem of Federated Semi-Supervised Learning (FSSL). In this work, we study this new problem of semi-supervised learning under federated learning framework, and propose a novel method to tackle it, which we refer to as Federated Matching (FedMatch). FedMatch improves upon naive federated semi-supervised learning approaches with a new inter-client consistency loss and decomposition of the parameters into parameters for labeled and unlabeled data. Through extensive experimental validation of our method in two different scenarios, we show that our method outperforms both local semi-supervised learning and baselines which naively combine federated learning with semi-supervised learning.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Murali Annavaram
- Qiang Yang
- Chaoyang He
- Yang Liu
- S. Avestimehr
- Praneeth Vepakomma
- Abhishek Singh
- Ramesh Raskar
Citations in the corpus
- FedML: A Research Library and Benchmark for Federated Machine Learning (2020, background): “In terms of exchanged information, aside from exchanging gradients and models, recent FL algorithms propose to exchange information such as pseudo labels in semi-supervised FL [56] and architecture parameters in neural architecture search-based FL [5, 57, 58].”
- Learn from Others and Be Yourself in Heterogeneous Federated Learning (2022, methodology): “We compare our method, FCCL with state-of-the-art approaches including FedDF [50], FML [72], FedMD [38], RCFL [16] and FedMatch [28].”