Natural Gradient Deep Q-learning
By Ethan Knight, Osher Lerner. 2018
10 citations recorded by the source corpus.
This paper presents findings for training a Q-learning reinforcement learning agent using natural gradient techniques. We compare the original deep Q-network (DQN) algorithm to its natural gradient counterpart (NGDQN), measuring NGDQN and DQN performance on classic controls environments without target networks. We find that NGDQN performs favorably relative to DQN, converging to significantly better policies faster and more frequently. These results indicate that natural gradient could be used for value function optimization in reinforcement learning to accelerate and stabilize training.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Joshua Achiam
- Sergey Levine
- Pieter Abbeel
- Ethan Knight
- Chelsea Finn
- Abhishek Gupta
- Han-Jia Ye
- Trevor Darrell
Citations in the corpus
- Towards Characterizing Divergence in Deep Q-Learning (2019, background): “…by Eqs 22 and 23 can be written as By the following lemma, the two updates in Eqs 27 and 28 are equivalent: The connection between NGQL and approximately non-expansive Q -update operators may explain the finding by (Knight & Lerner, 2018) that NGQL did not require target networks to remain stable.”