Secure Federated Transfer Learning

By Yang Liu, Tianjian Chen, Qiang Yang. 2018

128 citations recorded by the source corpus.

Machine learning relies on the availability of a vast amount of data for training. However, in reality, most data are scattered across different organizations and cannot be easily integrated under many legal and practical constraints. In this paper, we introduce a new technique and framework, known as federated transfer learning (FTL), to improve statistical models under a data federation. The federation allows knowledge to be shared without compromising user privacy, and enables complimentary knowledge to be transferred in the network. As a result, a target-domain party can build more flexible and powerful models by leveraging rich labels from a source-domain party. A secure transfer cross validation approach is also proposed to guard the FTL performance under the federation. The framework requires minimal modifications to the existing model structure and provides the same level of accuracy as the non-privacy-preserving approach. This framework is very flexible and can be effectively adapted to various secure multi-party machine learning tasks.

Subfields: Federated & Distributed Learning, Privacy & Security

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Federated & Distributed Learning

In Privacy & Security

Citations in the corpus

Advances and Open Problems in Federated Learning (2019, neutral): “Currently proposed algorithms include trees [103], linear and logistic regression [419, 198], and neural networks [276].”
Backdoor attacks and defenses in feature-partitioned collaborative learning (2020, background): “…collaborative learning frameworks have been developed for models including trees (Cheng et al., 2019), linear and logistic regression (Yang et al., 2019b; Hardy et al., 2017; Gratton et al., 2018; Kikuchi et al., 2017; Hu et al., 2019a), and neural networks (Liu et al., 2018; Hu et al., 2019b).”
A Communication Efficient Vertical Federated Learning Framework (2019, positive): “The objective is to perform a federated transfer learning (FTL) studied in [19].”
Real-World Image Datasets for Federated Learning (2019, methodology): “Federated learning algorithms are also applied to industrial scenarios including next-word-prediction [Hard et al. , 2018], speech keyword spot-ting [Leroy et al. , 2018], and images classiﬁcation [Liu et al. , 2018].”
Federated Multi-view Matrix Factorization for Personalized Recommendations (2020, unknown): “Two matrices: For two data sets, with partially paired samples, vertical federated learning approaches [24, 25] take advantage of the common samples, while horizontal federated learning approaches [1, 26] leverage the overlap of feature columns to improve the predictive performances.”
A Survey on the Convergence of Edge Computing and AI for UAVs: Opportunities and Challenges (2022, background): “3) Remaining Research Questions: a) How to integrate more homomorphic encryption (e.g., Liu et al. [161] integrated homomorphic encryption with federated learning for privacy protection).”
FATE: An Industrial Grade Platform for Collaborative Learning With Data Protection (2021, methodology): “, 2021)), FTL (Liu et al., 2018) and a variety of feature engineering tools such as feature binning, feature Information Value (IV) computations which are essential to real-world applications.”
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection (2019, background): “Specifically, researchers try to support more machine learning models with different privacy-preserving approaches, including deep neural networks (NNs) [22], [110], [119], [145], [196], gradient boosteddecision trees (GBDTs) [36], [95], [200], logistics regression [34], [128] and support vector mac”
A Survey on Federated Learning for Resource-Constrained IoT Devices (2021, unknown): “Liu et al. [28] designed a framework that can learn a feature representation of multiple parties based on common instances.”
Federated Neural Architecture Search (2020, background): “Such application scenarios are abounding: next-word prediction [16, 30], speech keyword spotting [22], image classification [27], etc.”
Practical Federated Gradient Boosting Decision Trees (2019, methodology): “The previous study (Du, Han, and Chen 2004) proposed a 2-party security model, which is also adopted in the previous studies (e.g., (Liu, Chen, and Yang 2018)).”
Privacy Preserving Machine Learning with Homomorphic Encryption and Federated Learning (2021, background): “There is also some work focusing on federated transfer learning, such as the framework designed in [36], that can be ﬂexibly applied to various secure multi-party machine learning.”
Federated Learning for Internet of Things: A Comprehensive Survey (2021, background): “FTL transfers features from different feature spaces to the same representation that is used to train data aggregated from multiple clients.”
A Survey on Federated Learning: The Journey From Centralized to Distributed On-Site Learning and Beyond (2021, background): “Federated transfer learning is proposed in [108] to build a privacy-preserved FL framework.”
Federated Machine Learning: Survey, Multi-Level Classification, Desirable Criteria and Future Directions in Communication and Networking Systems (2021, background): “In [66], the authors extend and improve upon [65] in two ways (1) reducing the overhead of”
Federated User Representation Learning (2019, background): “Personalization in FL poses many challenges due to its distributed nature, high communication costs, and privacy constraints [17, 5, 6, 19, 18, 20, 32, 12].”
A Survey on federated learning* (2020, methodology): “Liu et al. employed additively HE to modify neural network model and minimize the impact on training accuracy [18].”
Neural Architecture Search over Decentralized Data (2020, background): “Such application scenarios are abounding: next-word prediction [14, 24], speech keyword spotting [18], image classification [21], etc.”
FedVision: An Online Visual Object Detection Platform Powered by Federated Learning (2020, methodology): “In subsequent work, we will continue to incorporate more advanced FL algorithms (e.g. federated transfer learning (Liu et al. 2019; Gao et al. 2019)) into the platform to deal with more complex learning tasks.”
BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning (2020, methodology): “Many recent works [13, 37, 38,48] advocate the use of additively HE schemes, notably Paillier [46], as the primary means of privacy guarantee in cross-silo FL: each client transfers the encrypted local updates to the server for direct aggregation; the result is then sent back to each client for…”