Analysis of monotone gradient methods

By Yuhong Dai, Ya-Xiang Yuan. 2005

67 citations recorded by the source corpus.

The gradient method is one simple method in nonlinear optimization. In this paper, we give a brief review on monotone gradient methods and study their numerical properties by introducing a new technique of long-term observation. We find that, one monotone gradient algorithm which is proposed by Yuan recently shares with the Barzilai-Borwein (BB) method the property that the gradient components with respect to the eigenvectors of the function Hessian are decreasing together. This might partly explain why this algorithm by Yuan is comparable to the BB method in practice. Some examples are also provided showing that the alternate minimization algorithm and the other algorithm by Yuan may fall into cycles. Some more efficient gradient algorithms are provided. Particularly, one of them is monotone and performs better than the BB method in the quadratic case.

Subfields: Computational Mechanics, Computational Theory and Mathematics, Numerical Analysis

View Paper: https://www.semanticscholar.org/paper/868eedfe625a3080a16dbc800986bad7bc047708

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. G. Frassoldati
2. Gaetano Zanghirati
3. Luca Zanni

In Computational Theory and Mathematics

1. G. Frassoldati
2. Gaetano Zanghirati
3. Luca Zanni

In Numerical Analysis

1. G. Frassoldati
2. Gaetano Zanghirati
3. Luca Zanni

Citations in the corpus

• New adaptive stepsize selections in gradient methods (2008, methodology): “Starting from the new formula, Dai and Yuan [12] suggested a gradient method whose stepsize is given by”