Visualizing Dynamics: from t-SNE to SEMI-MDPs

By Nir Ben-Zrihem, Tom Zahavy, Shie Mannor. 2016

13 citations recorded by the source corpus.

Deep Reinforcement Learning (DRL) is a trending field of research, showing great promise in many challenging problems such as playing Atari, solving Go and controlling robots. While DRL agents perform well in practice we are still missing the tools to analayze their performance and visualize the temporal abstractions that they learn. In this paper, we present a novel method that automatically discovers an internal Semi Markov Decision Process (SMDP) model in the Deep Q Network's (DQN) learned representation. We suggest a novel visualization method that represents the SMDP model by a directed graph and visualize it above a t-SNE map. We show how can we interpret the agent's policy and give evidence for the hierarchical state aggregation that DQNs are learning automatically. Our algorithm is fully automatic, does not require any domain specific knowledge and is evaluated by a novel likelihood based evaluation criteria.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Management Science and Operations Research

Citations in the corpus

Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning (2018, background): “This success is mainly attributed to the power of deep neural networks to learn rich domain representations for approximating the value function or policy (Mnih et al., 2015; Zahavy, Ben-Zrihem, and Mannor, 2016; Zrihem, Zahavy, and Mannor, 2016).”