Proximal Algorithms

By Neal Parikh, Stephen P. Boyd. 2013

4,787 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computational Theory and Mathematics, Numerical Analysis, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computational Theory and Mathematics

In Numerical Analysis

In Optimization & Training

Citations in the corpus

Unsupervised Feature Learning via Non-parametric Instance Discrimination (2018, methodology): “e classes becomes infeasible. We tackle this challenge by approximating the full softmax distribution with noise-contrastive estimation (NCE) [9], and by resorting to a proximal regularization method [29] to stabilize the learning process. To evaluate the effectiveness of unsupervised learning, past”
Train faster, generalize better: Stability of stochastic gradient descent (2015, background): “h the indicator of the associated set. Soft-thresholding is the proximal point operator associated with the ℓ1-norm. For more information, see the surveys by Combettes and Wajs [6] or Parikh and Boyd [32]. An elementary proof of the following Lemma, due to Rockafellar [35], can be found in the appen”
Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms (2018, methodology): “…iterations, Elastic-averaging SGD (EASGD) proposed in (Zhang et al., 2015) adds a proximal term to the objective function in order to allow some slack between the models – an idea that is drawn from the Alternating Direction Method of Mul-a tipliers (ADMM) (Boyd et al., 2011; Parikh & Boyd, 2014).”
A Survey on Multi-Task Learning (2017, methodology): “Proximal method [118]: For a nonsmooth objective function, which is the sum of smooth and nonsmooth”
Partially Interpretable Estimators (PIE): Black-Box-Refined Interpretable Machine Learning (2021, methodology): “(12) Because of the nonsmooth of the grouped lasso penalty term, we develop an algorithm based on the proximal gradient descent (Parikh & Boyd, 2014).”
Generalized Sparse Additive Models (2019, methodology): “Here we give a general algorithm for ﬁtting GSAM s based on proximal gradient descent [Parikh and Boyd, 2014].”
Continual Learning with Node-Importance based Adaptive Group Sparse Regularization (2020, methodology): “Finally, from the theory of PGD [23], (7) is guaranteed to converge to a local minima of L t ( θ ) with appropriate α .”
Learning the Number of Neurons in Deep Networks (2016, methodology): “3, we follow a proximal gradient descent approach [Parikh and Boyd, 2014].”
A Universal Catalyst for First-Order Optimization (2015, background): “The catalyst acceleration can be interpreted as a variant of the proximal point algorithm [3, 9], which is a central concept in convex optimization, underlying augmented Lagrangian approaches, and composite minimization schemes [5, 20].”
Data-Driven Sparse Structure Selection for Deep Neural Networks (2017, methodology): “As show in Table 6, the performance of our ResNet-32-APG is better than ResNet-32-SGD.”
Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting (2015, background): “The classical deterministic proximal gradient approach [2], [36], [37] to solving (1) is to form a sequence via”
Auto-Sizing Neural Networks: With Applications to n-gram Language Models (2015, methodology): “Most work on learning with regularizers, including this work, can be thought of as instances of the proximal gradient method (Parikh and Boyd, 2014).”