Introduction to derivative-free optimization

By J. L. Nazareth. 2010

1,063 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming (2013, methodology): “There exists a somewhat long history for the development of zeroth-order (or derivative-free) methods in nonlinear programming (see the monograph by Conn et al. [5] and references therein).”
Google Vizier: A Service for Black-Box Optimization (2017, methodology): “More modern variants of these algorithms maintain simple models of the objective f within a subset of the feasible regions (called the trust region), and select a point xt to improve the model within the trust region [7].”
Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems (2012, neutral): “3 is inspired by gradient-free optimization, a topic extensively studied since [118, 147] — see [31, 67, 138, 140] for recent accounts on this theory.”
Automated Machine Learning: From Principles to Practices (2018, methodology): “Likewise, within the realm of oncology predictive modeling, an AutoML-based tool, JADBio [137], has been introduced to generating predictive models with high-performance criteria.”
Smoothing methods for nonsmooth, nonconvex minimization (2012, methodology): “Numerical methods for solving nonsmooth, nonconvex optimization problems have been studied extensively [7,10,12,35,37,51,59,68,77,102].”
Critical Hyper-Parameters: No Random, No Cry (2017, methodology): “Published methods for hyper-parameters search include evolution strategies [2], Gaussian processes [10, 23], pattern search [4], grid sampling and random sampling [1].”
Theoretical connections between optimization algorithms based on an approximate gradient (2013, background): “When adjoint code for computing the gradient of the cost functional to be maximized is not available, one has to seek alternative derivative-free optimization (DFO) algorithms [8].”
Complexity and global rates of trust-region methods based on probabilistic models (2018, methodology): “Conn, Scheinberg, and Vicente [9, 10, 12] provided a systematic approach to the subject of deterministic sampling geometry in DFO, establishing error bounds for polynomial interpolation and regression models in terms of a constant measuring the quality or well poisedness of the corresponding sample ”
Taking Human out of Learning Applications: A Survey on Automated Machine Learning (2018, methodology): “Optimization from samples [77] is a kind of smarter search approach compared with simple ones in Section 4.”
A Stochastic Simplex Approximate Gradient (StoSAG) for optimization under uncertainty (2017, methodology): “Do and Reynolds [15] provided an analysis of the deterministic EnOpt method and demonstrated its close connection to other stochastic gradient estimation techniques such as simultaneous perturbation stochastic approximation (SPSA, [16, 17]) and the simplex gradient method [18].”
Learning to Navigate the Energy Landscape (2016, unknown): “We refer the reader to [8] for extensive discussions on derivative-free optimization.”
Geometry of sample sets in derivative-free optimization: polynomial regression and underdetermined interpolation (2008, methodology): “A formal, theoretically supported method for interpolation sets based on Lagrange polynomials is described in [9] and can be extended to the regression case.”
Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator (2018, methodology): “Due to the Gaussian smoothing, the gradient has the particularly simple functional form (see Conn et al. (2009); Nesterov & Spokoiny (2015)): .”
Query Complexity of Derivative-Free Optimization (2012, background): “However, there are algorithms with upper bounds on the rates of convergence for stochastic DFO with the function evaluation oracle [15, 16].”
Direct Multisearch for Multiobjective Optimization (2011, methodology): “In this paper, we are interested in the development of derivative-free methods (see [9]) for Multiobjective optimization (MOO).”
Worst case complexity of direct search (2013, methodology): “Thus, under the assumption that the objective function f is bounded from below, it is possible to prove that there exists a subsequence of unsuccessful iterates driving the step size parameter to zero (see [13] or [7, Theorems 7.1 and 7.11 and Corollary 7.2]).”