Exploiting Unintended Feature Leakage in Collaborative Learning

By Luca Melis, Congzheng Song, Emiliano De Cristofaro, Vitaly Shmatikov. 2018

1,798 citations recorded by the source corpus.

Collaborative machine learning and related techniques such as federated learning allow multiple participants, each with his own training dataset, to build a joint model by training locally and periodically exchanging model updates. We demonstrate that these updates leak unintended information about participants' training data and develop passive and active inference attacks to exploit this leakage. First, we show that an adversarial participant can infer the presence of exact data points -- for example, specific locations -- in others' training data (i.e., membership inference). Then, we show how this adversary can infer properties that hold only for a subset of the training data and are independent of the properties that the joint model aims to capture. For example, he can infer when a specific person first appears in the photos used to train a binary gender classifier. We evaluate our attacks on a variety of tasks, datasets, and learning configurations, analyze their limitations, and discuss possible defenses.

Subfields: Artificial Intelligence, Health Informatics, Federated & Distributed Learning, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Qiang Yang
  2. Percy Liang
  3. Yang Liu
  4. Philip S. Yu
  5. Quoc V. Le
  6. Martin Jaggi
  7. Dacheng Tao
  8. Yoshua Bengio

In Health Informatics

  1. Lingjuan Lyu
  2. Qiang Yang
  3. Congzheng Song
  4. Amir Houmansadr
  5. Reza Shokri
  6. Murali Annavaram
  7. Hamed Haddadi
  8. H. Brendan McMahan

In Federated & Distributed Learning

  1. Qiang Yang
  2. Virginia Smith
  3. Jianyu Wang
  4. Gauri Joshi
  5. Jakub Konecný
  6. Tian Li
  7. Zachary B. Charles
  8. Yang Liu

In Privacy & Security

  1. Peter Kairouz
  2. Abhradeep Thakurta
  3. Qiang Yang
  4. Om Thakkar
  5. H. Brendan McMahan
  6. Lingjuan Lyu
  7. Badih Ghazi
  8. Vitaly Shmatikov

Citations in the corpus