Simplifying Deep Reinforcement Learning via Self-Supervision
By Daochen Zha, Kwei-Herng Lai, Kaixiong Zhou, Xia Hu. 2021
16 citations recorded by the source corpus.
Supervised regression to demonstrations has been demonstrated to be a stable way to train deep policy networks. We are motivated to study how we can take full advantage of supervised loss functions for stably training deep reinforcement learning agents. This is a challenging task because it is unclear how the training data could be collected to enable policy improvement. In this work, we propose Self-Supervised Reinforcement Learning (SSRL), a simple algorithm that optimizes policies with purely supervised losses. We demonstrate that, without policy gradient or value estimation, an iterative procedure of ``labeling"data and supervised regression is sufficient to drive stable policy improvement. By selecting and imitating trajectories with high episodic rewards, SSRL is surprisingly competitive to contemporary algorithms with more stable performance and less running time, showing the potential of solving reinforcement learning with supervised learning techniques. The code is available at https://github.com/daochenzha/SSRL
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
Citations in the corpus
- TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations (2021, positive): “…of the designated player: where u i build_in is represented for the build-in action and I ( i, t ) is an indicator function, which is defined as: Using past good experiences as the supervision has shown great success in many control tasks (Oh et al., 2018; Vinyals et al., 2019; Zha et al., 2021).”