Stochastic Alternating Direction Method of Multipliers

By Hua Ouyang, Niao He, Long Tran, Alexander G. Gray. 2013

300 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning with Elastic Averaging SGD (2014, background): “The linearization of the penalty term leads to the update rule for the center ˜ x of the form of moving average where the average is taken over both space and time.”
Combining the benefits of function approximation and trajectory optimization (2014, background): “It would be interesting to adapt recent work on stochastic ADMM [21] to this problem.”
Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization (2014, background): “Several authors [42, 28, 37, 49] have considered a more general formulation than (26), where each φi is a function of the whole vector z ∈ Rn.”
Trading Computation for Communication: Distributed Stochastic Dual Coordinate Ascent (2013, methodology): “For ADMM-based algorithms, we implement a stochastic ADMM in [14] (ADMM-s) and a deterministic ADMM in [23] (ADMM-dca) that employes the DCA algorithm for solving the subproblems.”
Group-Based Alternating Direction Method of Multipliers for Distributed Linear Classification (2017, methodology): “In the other class, the alternating direction method of multipliers (ADMM) [14] is one of the most well-known algorithms.”
Towards an optimal stochastic alternating direction method of multipliers (2014, methodology): “1 one obtains the SADMM method of (Ouyang et al., 2013).”
Unwrapping ADMM: Efficient Distributed Computing via Transpose Reduction (2015, background): “Many variations of ADMM have subsequently been proposed, including specialized variants for decentralized systems [10], asynchronous updates [11, 12], inexact solutions to subproblems [13], and online/stochastic updates [12].”
Asynchronous Distributed ADMM for Consensus Optimization (2014, methodology): “As shown in (Ouyang et al., 2013; Suzuki, 2013), this is more efficient in this context than other state-of-the-art solvers.”
DP-ADMM: ADMM-Based Distributed Learning With Differential Privacy (2018, background): “Stochastic ADMM [39], [40] considers stochastic and composite objective functions caused by natural uncertainties in observations.”
iPrivacy: Image Privacy Protection by Identifying Sensitive Objects via Deep Multi-Task Learning (2017, methodology): “Then, a CRF model is further learned to integrate the neighboring pixels for the same object classes to generate semantic object regions.”
A Coordinate Descent Primal-Dual Algorithm and Application to Distributed Asynchronous Optimization (2014, methodology): “In Theorem 5 below, we apply this result to the operator related to ADMM+, and develop an asynchronous version of ADMM+ in the context where it is distributed on a graph.”
Stochastic Dual Coordinate Ascent with Alternating Direction Method of Multipliers (2014, methodology): “Recently some online variants of ADMM have been proposed by Wang and Banerjee (2012); Suzuki (2013); Ouyang et al. (2013).”
Adaptive Consensus ADMM for Distributed Optimization (2017, background): “In theory, ADMM converges for any constant penalty parameter (Eckstein & Bertsekas, 1992; He & Yuan, 2012; Ouyang et al., 2013).”
Distributed stochastic optimization for deep learning (thesis) (2016, unknown): “Note that since the step size for the dual ascent update is chosen to be ρ by convention [11, 61, 42], we have re-parametrized the Lagrangian multiplier to be λt ← λt/ρ in the above updates.”