CONVERGENCE CONDITIONS FOR ASCENT METHODS

By Siam Rfview. 2016

507 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/3e1fc030e42ee2203e51ca12f1180ae2b7ecfee9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Guaranteed Sufficient Decrease for Stochastic Variance Reduced Gradient Optimization (2018, methodology): “, the wellknown line search technique [28]) has been studied for deterministic optimization [25, 26].”
Large-scale distributed L-BFGS (2017, methodology): “In this method, αk is chosen to satisfy the Wolfe Condition [29] so there is no need to manually select a constant value for αk.”
Pretraining Convolutional Neural Networks for Visual Recognition (2016, methodology): “However, using the iterative scheme, the algorithm is guaranteed to find a local minimum for convex functions when sufficiently small step size is used [41].”
The Nonlinearity Coefficient - Predicting Overfitting in Deep Neural Networks (2018, background): “In yet other words, as the GSC increases, the relative size of the region that satisfies the first Wolfe condition shrinks proportionally [37].”
Probabilistic Line Searches for Stochastic Optimization (2015, methodology): “The Wolfe [21] conditions are a widely accepted formalization of this notion; they consider t acceptable if it fulfills f(t) ≤ f(0) + c1tf ′(0) (W-I) and f ′(t) ≥ c2f ′(0) (W-II), (2) using two constants 0 ≤ c1 < c2 ≤ 1 chosen by the designer of the line search, not the user.”
Backtracking gradient descent method for general $C^1$ functions, with applications to Deep Learning (2018, background): “There are some abstract conditions on functions with only directional derivatives (see the deﬁnition about serious steps on page 228 in [47]), under which convergence results can be proven.”