Eavesdrop the Composition Proportion of Training Labels in Federated Learning

By Lixu Wang, Shichao Xu, Xiao Wang, Qi Zhu. 2019

74 citations recorded by the source corpus.

Federated learning (FL) has recently emerged as a new form of collaborative machine learning, where a common model can be learned while keeping all the training data on local devices. Although it is designed for enhancing the data privacy, we demonstrated in this paper a new direction in inference attacks in the context of FL, where valuable information about training data can be obtained by adversaries with very limited power. In particular, we proposed three new types of attacks to exploit this vulnerability. The first type of attack, Class Sniffing, can detect whether a certain label appears in training. The other two types of attacks can determine the quantity of each label, i.e., Quantity Inference attack determines the composition proportion of the training label owned by the selected clients in a single round, while Whole Determination attack determines that of the whole training process. We evaluated our attacks on a variety of tasks and datasets with different settings, and the corresponding results showed that our attacks work well generally. Finally, we analyzed the impact of major hyper-parameters to our attacks and discussed possible defenses.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Federated Class-Incremental Learning (2022, background): “Specifically, inspired by [45, 46], for a single sample (xli,y t li) (its ground-truth”
Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges (2022, unknown): “[78] proposes a set of passive attacks.”
Non-Transferable Learning: A New Approach for Model Ownership Verification and Applicability Authorization (2021, background): “For instance, Inference Attack (Shokri et al., 2017; Wang et al., 2019) can steal private information about the target DL model.”
Addressing Class Imbalance in Federated Learning (2020, background): “Despite of various types of inference attacks (Melis et al. 2018; Zhu, Liu, and Han 2019; Wang et al. 2019; Sun et al. 2020), inferring information of particular clients is still extremely difficult.”
A Comprehensive Survey of Privacy-preserving Federated Learning (2021, background): “WHO RA1 [61, 196, 228] [26, 137, 218] [157, 169, 198] [144, 164, 199] [63, 70, 193] [16, 174, 191] [78, 110, 228] [52, 81, 216] [32, 56, 222] [30, 127, 173] RA2 [114, 175, 187] [215, 221, 224] [29, 111, 113] [13, 199] [3, 10, 40] [109, 155, 168] [45, 172, 193] [20, 52, 57] [56, 165, 197] [173, 199, ”
FLAME: Taming Backdoors in Federated Learning (2022, background): “A number of attacks on FL have been proposed that aim to infer from parameters of a model the presence of a specific training sample in the training dataset (membership inference attacks) [41, 46, 52], properties of training samples (property inference attacks) [23,41], try to assess the proportion ”
Federated Learning for Internet of Things: A Comprehensive Survey (2021, background): “As a result, malicious clients and attacks can be detected and eliminated from the FL process while the federated model is preserved [79].”
The Limitations of Federated Learning in Sybil Settings (2020, background): “For example, adversaries posing as honest clients can send erroneous updates to poison the trained model [3,8,15,19,23,28,36], invert or reconstruct the data of honest clients [26,41,42,48,54,56], or gain access to models without usefully contributing [34].”