Optimizing Costly Functions with Simple Constraints: A Limited-Memory Projected Quasi-Newton Algorithm

By Mark W. Schmidt, Ewout van den Berg, Michael P. Friedlander, Kevin P. Murphy. 2009

283 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

DAGs with NO TEARS: Continuous Optimization for Structure Learning (2018, methodology): “More recently, extremely eﬃcient algorithms have been developed for this problem using coordinate descent (Friedman et al., 2008) and Newton methods (Hsieh et al., 2014; Schmidt et al., 2009).”
Sub-Sampled Newton Methods I: Globally Convergent Algorithms (2016, background): “The results of Theorems 3, 4, 5, and 8 give global convergence of the corresponding algorithms with inexact updates.”
Efficient Dimensionality Reduction for High-Dimensional Network Estimation (2014, methodology): “(2009) and Marlin & Murphy (2009) make use of these methods (Duchi et al., 2008; Schmidt et al., 2009), after first identifying the groups of the variables when the modular structure is unknown.”
Efficient Online and Batch Learning Using Forward Backward Splitting (2009, unknown): “Furthermore, the sparse Fobos updates apply equally as well to mixed norm regularization, and while there are efficient algorithms for both projection to both l1/l2 and l1/l∞ balls (Schmidt et al., 2009; Quattoni et al., 2009), they are more complicated than the Fobos steps.”
Speech enhancement based on deep denoising autoencoder (2013, methodology): “In this study, a linear search based quasi-Newton optimization algorithm is used to estimate (W∗,b∗, c∗) [11].”
Sparse inverse covariance matrix estimation using quadratic approximation (2011, methodology): “Schmidt et al. [2009] propose a second order Projected Quasi-Newton method ( PQN ) that solves the dual problem (4), since the dual objective function is smooth.”
Convex Structure Learning in Log-Linear Models: Beyond Pairwise Potentials (2010, methodology): “The SPG algorithm may be useful for other problems with overlapping group `1-regularization, while Dykstra's algorithm could alternately be used within the optimal method of Nesterov (2003) or projected quasiNewton methods (Schmidt et al., 2009).”
Modeling Discrete Interventional Data using Directed Cyclic Graphical Models (2009, methodology): “Since the edge parameters form disjoint sets, projection onto these constraints is a trivial computation (Boyd and Vandenberghe, 2004, Exercise 8.3(c)), and the optimization problem can be efficiently solved using a limited-memory projected quasi-Newton method (Schmidt et al., 2009)”
Deep Learning for Predicting Human Strategic Behavior (2016, methodology): “[25]’s implementation of the simplex projection operation to optimise our model.”
Second-order Conditional Gradients (2020, methodology): “…to tackle this problem, such as interior point methods (Yuan & Lin, 2007), block coordinate descent or accelerated ﬁrst-order algorithms (Banerjee et al., 2008), coordinate descent algorithms (Friedman et al., 2008) and even projected limited-memory quasi-Newton algorithms (Schmidt et al., 2009).”
Convergent Decomposition Solvers for Tree-reweighted Free Energies (2011, methodology): “Convergence follows from the analysis of the SPG method by Wang et al. (2005) The projected quasi-Newton (PQN) method was recently introduced by Schmidt et al. (2009) and can be considered a generalization of L-BFGS (Nocedal, 1980) to constrained optimization.”
11 Projected Newton-type Methods in Machine Learning (2011, methodology): “This generalization is similar to those examined in (Duchi et al., 2008; Schmidt et al., 2009), and it encourages the Gaussian graphical model to be sparse across groups of variables (i.”
Generative-Discriminative Basis Learning for Medical Imaging (2012, result): “Similar approaches have been taken recently by Schmidt et al. [42] and Wright et al. [43] for large-scale non-smooth problems.”