BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems

By Zachary Chase Lipton, Xiujun Li, Jianfeng Gao, Lihong Li, Faisal Ahmed, Li Deng. 2016

173 citations recorded by the source corpus.

We present a new algorithm that significantly improves the efficiency of exploration for deep Q-learning agents in dialogue systems. Our agents explore via Thompson sampling, drawing Monte Carlo samples from a Bayes-by-Backprop neural network. Our algorithm learns much faster than common exploration strategies such as ε-greedy, Boltzmann, bootstrapping, and intrinsic-reward-based ones. Additionally, we show that spiking the replay buffer with experiences from just a few successful episodes can make Q-learning feasible when it might otherwise fail.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning, Dialogue & Conversational AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Sergey Levine
  2. Yoshua Bengio
  3. Pieter Abbeel
  4. Jianfeng Gao
  5. Mohit Bansal
  6. Ruslan Salakhutdinov
  7. Percy Liang
  8. Quoc V. Le

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Xiaogang Wang
  5. Ming-Hsuan Yang
  6. Andrew Zisserman
  7. Jitendra Malik
  8. Andrea Vedaldi

In Reinforcement Learning

  1. Sergey Levine
  2. Pieter Abbeel
  3. Yuxi Li
  4. Emma Brunskill
  5. Marc G. Bellemare
  6. R´emi Munos
  7. Joelle Pineau
  8. Hado van Hasselt

In Dialogue & Conversational AI

  1. Minlie Huang
  2. Charles Blundell
  3. Anima Anandkumar
  4. Emma Brunskill
  5. Kamyar Azizzadenesheli
  6. Meire Fortunato
  7. Xiaoyan Zhu
  8. Oriol Vinyals

Citations in the corpus