TRUST REGION METHODS

By David F. Gleich. 2017

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Off-Policy Actor-Critic with Shared Experience Replay (2019, methodology): “Furthermore we propose a trust region scheme (Conn et al., 2000; Schulman et al., 2015; 2017) in Section 4 that enables efﬁcient learning even in a strongly off-policy regime, where distinct agents share the experience replay module and learn from each others experience.”
New Insights and Perspectives on the Natural Gradient Method (2014, methodology): “In this view, λ plays the role of a Tikhonov damping parameter (Tikhonov, 1943; Conn et al., 2000; Nocedal and Wright, 2006; Martens and Sutskever, 2012) which is added to Bk in order to ensure that the proposed update stays within a certain region around zero in which Mk(δ) remains a reasonable…”
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization (2019, methodology): “This is also known as the trust-region type iteration in existing optimization literature (Conn et al., 2000).”
Global rates of convergence for nonconvex optimization on manifolds (2016, unknown): “There exists c 3 > 0 such that, for all k such that grad f ( x k ) ≤ ε g and This can be achieved by making a step of maximal length along a direction which certiﬁes that λ min ( H k ) − ε H (Conn et al., 2000): this is called an eigenstep .”
Adaptive norms for deep learning with regularized Newton methods (2019, methodology): “It is precisely this Hessian information that allows regularized Newton methods to enjoy superlinear local convergence as well as to provably escape saddle points (Conn et al., 2000).”
HOnnotate: A Method for 3D Annotation of Hand and Object Poses (2019, methodology): “In the multi-camera setup, we obtain a ﬁrst estimate ˜ p h for the hand pose in the ﬁrst frame ♣ t ✏ 0 q as, We use the Dogleg optimizer [11] to perform this optimization.”
Ellipsoidal Trust Region Methods and the Marginal Value of Hessian Information for Neural Network Training (2019, methodology): “Among the class of regularized Newton methods, trust region [22] and cubic regularization algorithms [16] are the most principled approaches, as they yield the strongest convergence guarantees.”
A Geometric Analysis of Phase Retrieval (2016, background): “In contrast, our global geometric analysis allows flexible algorithm design (i.e., separation of geometry and algorithms) and gives some clues as to the behavior of nonconvex heuristics used in practice, which often succeed without clever initialization.”
Convergence Rate Analysis of a Stochastic Trust-Region Method via Supermartingales (2019, methodology): “We now state and analyze a stochastic trust region (TR) algorithm (Algorithm 1) which is essentially very similar to its deterministic counterpart [9].”
Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods (2019, methodology): “Such first-order optimality measure has been used before in the context of optimization; see [9].”
Numerical optimization (2004, methodology): “The monograph of Conn, Gould, and Toint [74] is an exhaustive treatment of the state of the art in trust-region methods for both unconstrained and constrained optimization.”
A Distributed Second-Order Algorithm You Can Trust (2018, methodology): “In this paper, we instead leverage ideas from trust-region methods (Conn et al., 2000), where we dynamically adapt the model based on how much we trust the approximate second-order information.”