Two-Point Step Size Gradient Methods

By J. Barzilai, Jonathan Michael Borwein. 1988

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Interpretable Recurrent Neural Networks Using Sequential Sparse Recovery (2016, methodology): “…, such as fast ISTA (FISTA) [20], which combines Nesterov’s adaptive step size method [21] with a gradual decrease in λ ( k ) , and sparse reconstruction by separable approximation (SparSA) [22], which combines Barzilai-Borwein gradient step size adjustment [23] with a gradual decrease in λ ( k ) .”
Distributed optimization with arbitrary local solvers (2015, methodology): “CD Coordinate Descent [51] APPROX Accelerated, Parallel and Proximal Coordinate Descent [20, 21] GD Gradient Descent with Backtracking Line Search [44] CG Conjugate Gradient Method [25] L-BFGS Quasi-Newton with Limited-Memory BFGS Updating [10] BB Barzilai-Borwein Gradient Method [4] FISTA Fast Iter”
Stochastic, Distributed and Federated Optimization for Machine Learning (2017, methodology): “CD Coordinate Descent [148] APPROX Accelerated, Parallel and Proximal Coordinate Descent [59] GD Gradient Descent with Backtracking Line Search [134] CG Conjugate Gradient Method [77] L-BFGS Quasi-Newton with Limited-Memory BFGS Updating [27] BB Barzilai-Borwein Gradient Method [9] FISTA Fast Iterat”
Machine Learning a General-Purpose Interatomic Potential for Silicon (2018, background): “7 Å) to exclude unphysically close atoms, relaxed with the two-point steepest-descent [209] method.”
Automatic and Precise Orthorectification, Coregistration, and Subpixel Correlation of Satellite Images, Application to Ground Deformation Measurements (2007, methodology): “APPENDIX II TPSS ALGORITHM APPLIED TO THE INVERSE ORTHORECTIFICATION PROBLEM By calling Rx and Ry as the nominal image ground resolution in the Easting and Northing directions, the best image subpixel coordinates m = (x∗, y∗) that minimize Φ (7) are given by the following TPSS [21] algorithm: m−1 = ”
Barzilai-Borwein Step Size for Stochastic Gradient Descent (2016, methodology): “The BB method, proposed by Barzilai and Borwein in [2], has been proven to be very successful in solving nonlinear optimization problems.”
EGO-Planner: An ESDF-Free Gradient-Based Local Planner for Quadrotors (2020, unknown): “(12) and recursively expands for m steps and then yields the efficient two-loop recursion updating method [16], resulting in linear time/space complexity.”
A feasible method for optimization with orthogonality constraints (2012, methodology): “While our steepest descent Algorithm 1 is extremely simple, the well-known Barzilai-Borwein (BB) step size [5] is often able to accelerate the gradient method at nearly no extra cost.”
11 Projected Newton-type Methods in Machine Learning (2011, methodology): “In SPG methods, either α or β is set to one, and the other step size is set to one of the step sizes proposed by (Barzilai and Borwein, 1988).”
Big Batch SGD: Automated Inference using Adaptive Batch Sizes (2016, methodology): “We also derive an adaptive stepsize method based on the Barzilai and Borwein (1988) curvature estimate that fully automates the big batch method, while empirically enjoying a faster convergence rate than the Armijo backtracking line search.”
A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization (2004, background): “Since f is convex, f (x(t)) is a convex function of t, and the derivative f � (x(t)) is an increasing function of t ∈ [0, 1] with f � (x(0)) = 0. Hence, for t ∈ [0, 1 ], f � (x(t)) attains its maximum value at t = 1. This”
Fast L1-Minimization Algorithms For Robust Face Recognition (2010, unknown): “In [21], assuming α k = L f , one can show that ISTA has a sublinear convergence rate that is no worse than O (1 /k ) : Meanwhile, an alternative way of determining α k at each iteration is used in SpaRSA [25], which is based on the Barzilai-Borwein equation [57].”
Sparse Reconstruction by Separable Approximation (2008, methodology): “We are particularly interested in variants based on the formula proposed by Barzilai and Borwein (BB) [1] in the context of smooth nonlinear minimization; see also [19] , [50].”
Fast ℓ1-minimization algorithms and an application in robust face recognition: A review (2010, unknown): “This is the so-called Barzilai-Borwein equation [3], [39], [44].”
Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems (2007, methodology): “Recently, considerable atte ntion has been paid to an approach due to Barzilai and Borwein (BB) [2] that does not have this property.”
Blind deconvolution in scientific imaging & computational photography (2011, methodology): “This reparametrisation has a long history in image deconvolution (Biraud, 1969), but numerically, the ensuing nonlinearity can be damaging as it destroys the convexity of sub-problems. More recently, Matson et al. (2008) also used the same nonlinear (x→ z(2)) reparametrisation for solving MFBD with ”
Efficient multi-task feature learning with calibration (2014, methodology): “However, due to the utilization of the Barzilai-Borwein (BB) rule [3, 32], the empirical convergence performance of SpaRSA is much better than that of FISTA for solving the dual problem in Eq.”