Towards Better Interpretability in Deep Q-Networks
By Raghuram Mandyam Annasamy, Katia P. Sycara. 2018
69 citations recorded by the source corpus.
Deep reinforcement learning techniques have demonstrated superior performance in a wide variety of environments. As improvements in training algorithms continue at a brisk pace, theoretical or empirical studies on understanding what these networks seem to learn, are far behind. In this paper we propose an interpretable neural network architecture for Q-learning which provides a global explanation of the model’s behavior using key-value memories, attention and reconstructible embeddings. With a directed exploration strategy, our model can reach training rewards comparable to the state-of-the-art deep Q-learning models. However, results suggest that the features extracted by the neural network are extremely shallow and subsequent testing using out-of-sample examples shows that the agent can easily overfit to trajectories seen during training.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Jeff Clune
- Pieter Abbeel
- Ruslan Salakhutdinov
- Chelsea Finn
- Yoshua Bengio
- Joel Lehman
- Benjamin Eysenbach
Citations in the corpus
- Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning (2019, background): “The other two attempt to manipulate semantics in the pixel input to assess the agent’s response by, for example, adding an additional object to verify a hypothesis about memorization (Annasamy & Sycara, 2019).”
- Contrastive Learning as Goal-Conditioned Reinforcement Learning (2022, background): “However, prior work has found it challenging to uphold this promise [7, 79, 118, 125], prompting many prior methods to employ separate objectives for representation learning and RL [31, 43, 70, 74, 89, 91, 100, 101, 117, 140, 143].”
- An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents (2018, methodology): “Another approach is to modify the DNN architecture or training procedure such that a trained model will have more interpretable features [Annasamy and Sycara, 2018].”