Derivative-Free Optimization via Classification

By Yang Yu, Hong Qian, Yi-Qi Hu. 2016

96 citations recorded by the source corpus.

Many randomized heuristic derivative-free optimization methods share a framework that iteratively learns a model for promising search areas and samples solutions from the model. This paper studies a particular setting of such framework, where the model is implemented by a classification model discriminating good solutions from bad ones. This setting allows a general theoretical characterization, where critical factors to the optimization are discovered. We also prove that optimization problems with Local Lipschitz continuity can be solved in polynomial time by proper configurations of this framework. Following the critical factors, we propose the randomized coordinate shrinking classification algorithm to learn the model, forming the RACOS algorithm, for optimization in continuous and discrete domains. Experiments on the testing functions as well as on the machine learning tasks including spectral clustering and classification with Ramp loss demonstrate the effectiveness of RACOS.

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qiang Yang
2. Yuqiang Chen
3. Wei-Wei Tu
4. Quanming Yao
5. Wenyuan Dai
6. Yang Yu
7. Yi-Qi Hu
8. Zhi-Hua Zhou

In Management Science and Operations Research

1. Zhi-Hua Zhou
2. Yang Yu
3. Qiang Yang
4. Wei-Wei Tu
5. Yi-Qi Hu
6. Wenyuan Dai
7. Yuqiang Chen
8. Hugo Jair Escalante

Citations in the corpus

• Multi-Fidelity Automatic Hyper-Parameter Tuning via Transfer Series Expansion (2019, methodology): “We implement the proposed multi-fidelity framework with TSE based on classification-based optimization method SR ACOS (Yu, Qian, and Hu 2016; Hu, Qian, and Yu 2017), and name it T SE SR ACOS .”
• Tunneling Neural Perception and Logic Reasoning through Abductive Learning (2018, methodology): “RACOS is a randomized derivative-free optimization method implemented by a classification model that discriminates good solutions from bad ones, and it achieves good performance on complex optimization problems.”
• Experienced Optimization with Reusable Directional Model for Hyper-Parameter Search (2018, methodology): “The processes of derivative-free optimization algorithms [Shahriari et al. , 2015; Munos, 2011; Yu et al. , 2016] share the same idea of optimization from samples.”
• Taking Human out of Learning Applications: A Survey on Automated Machine Learning (2018, background): “• Classification-based optimization [39], [89]: Based on previous samples, classification-based optimization models search space by learning a classifier.”