Privacy-Preserving Stochastic Gradual Learning

By Bo Han, Ivor Wai-Hung Tsang, Xiaokui Xiao, Ling Chen, Sai-Fu Fung, Celina Ping Yu. 2018

9 citations recorded by the source corpus.

It is challenging for stochastic optimization to handle large-scale sensitive data safely. Duchi et al. recently proposed a private sampling strategy to solve privacy leakage in stochastic optimization. However, this strategy leads to a degeneration in robustness, since this strategy is equal to noise injection on each gradient, which adversely affects updates of the primal variable. To address this challenge, we introduce a robust stochastic optimization under the framework of local privacy, which is called Privacy-pREserving StochasTIc Gradual lEarning (PRESTIGE). PRESTIGE bridges private updates of the primal variable (by private sampling) with gradual curriculum learning (CL). The noise injection leads to similar issue from label noise, but the robust learning process of CL can combat with label noise. Thus, PRESTIGE yields “private but robust” updates of the primal variable on the curriculum, that is, a reordered label sequence provided by CL. In theory, we reveal the convergence rate and maximum complexity of PRESTIGE. Empirical results on six datasets show that PRESTIGE achieves a good tradeoff between privacy preservation and robustness over baselines.

Subfields: Artificial Intelligence, Computer Science Applications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Orthogonal Gradient Descent for Continual Learning (2019, background): “through the model prediction gradients not the actual data) is also related di↵erential private learning (Wu et al., 2017; Li et al., 2018; Pihur et al., 2018; Han et al., 2018) and federated learning (Bonawitz et al.”