Sequential Classification-Based Optimization for Direct Policy Search

By Yi-Qi Hu, Hong Qian, Yang Yu. 2017

51 citations recorded by the source corpus.

Classification-based optimization is a recently developed framework for derivative-free optimization, which has shown to be effective for non-convex optimization problems with many local optima. This framework requires to sample a batch of solutions for every update of the search model. However, in reinforcement learning, direct policy search often offers only sequential policy evaluation. Thus, classificationbased optimization is not efficient for direct policy search where solutions have to be sampled sequentially. In this paper, we adapt the classification-based optimization for sequential sampled solutions by forming the batch of reused historical solutions. Experiments on helicopter hovering control task and reinforcement learning benchmark tasks in OpenAI Gym show that the new algorithm is superior to state-of-the-art derivative-free optimization approaches.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qiang Yang
2. Xipeng Qiu
3. Xuanjing Huang
4. Yang Yu
5. Wei-Wei Tu
6. Yuqiang Chen
7. Yi-Qi Hu
8. Wenyuan Dai

In Computational Theory and Mathematics

1. Yang Yu
2. Qiang Yang
3. Hugo Jair Escalante
4. Isabelle M Guyon
5. Wenyuan Dai
6. Xipeng Qiu
7. Xuanjing Huang
8. Yu-Feng Li

In Management Science and Operations Research

1. Yang Yu
2. Yi-Qi Hu
3. Qiang Yang
4. Wei-Wei Tu
5. Wenyuan Dai
6. Yuqiang Chen
7. Tianxiang Sun
8. Hugo Jair Escalante

Citations in the corpus

• Multi-Fidelity Automatic Hyper-Parameter Tuning via Transfer Series Expansion (2019, methodology): “We implement the proposed multi-fidelity framework with TSE based on classification-based optimization method SR ACOS (Yu, Qian, and Hu 2016; Hu, Qian, and Yu 2017), and name it T SE SR ACOS .”
• Black-Box Tuning for Language-Model-as-a-Service (2022, background): “…2016), etc. Due to their ability of addressing complex optimization tasks, DFO algorithms have achieved many impressive applications in automatic machine learning (Snoek et al., 2012), reinforcement learning (Salimans et al., 2017; Hu et al., 2017), objective detection (Zhang et al., 2015b), etc.”
• Taking Human out of Learning Applications: A Survey on Automated Machine Learning (2018, background): “• Classification-based optimization [39], [89]: Based on previous samples, classification-based optimization models search space by learning a classifier.”