Reinforcement Learning for Learning Rate Control

By Chang Xu, Tao Qin, G. Wang, Tie-Yan Liu. 2017

36 citations recorded by the source corpus.

Stochastic gradient descent (SGD), which updates the model parameters by adding a local gradient times a learning rate at each step, is widely used in model training of machine learning algorithms such as neural networks. It is observed that the models trained by SGD are sensitive to learning rates and good learning rates are problem specific. We propose an algorithm to automatically learn learning rates using neural network based actor-critic methods from deep reinforcement learning (RL).In particular, we train a policy network called actor to decide the learning rate at each step during training, and a value network called critic to give feedback about quality of the decision (e.g., the goodness of the learning rate outputted by the actor) that the actor made. The introduction of auxiliary actor and critic networks helps the main network achieve better performance. Experiments on different datasets and network architectures show that our approach leads to better convergence of SGD than human-designed competitors.

Subfields: Artificial Intelligence, Reinforcement Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Yoshua Bengio
  3. Jitendra Malik
  4. Trevor Darrell
  5. Quoc V. Le
  6. Alexei A. Efros
  7. Mohit Bansal
  8. Luc Van Gool

In Reinforcement Learning

  1. Sergey Levine
  2. Jitendra Malik
  3. James Harrison
  4. Pieter Abbeel
  5. Jascha Narain Sohl-Dickstein
  6. Alexei A. Efros
  7. Clemens Winter
  8. Diogo Almeida

In Optimization & Training

  1. C. Freeman
  2. Luke Metz
  3. Niru Maheswaranathan
  4. Jascha Narain Sohl-Dickstein
  5. Alexei A. Efros
  6. Jitendra Malik
  7. Clemens Winter
  8. Diogo Almeida

Citations in the corpus