Towards Mixed Optimization for Reinforcement Learning with Program Synthesis

By Surya Bhupatiraju, Kumar Krishna Agrawal, Rishabh Singh. 2018

8 citations recorded by the source corpus.

Deep reinforcement learning has led to several recent breakthroughs, though the learned policies are often based on black-box neural networks. This makes them difficult to interpret and to impose desired specification constraints during learning. We present an iterative framework, MORL, for improving the learned policies using program synthesis. Concretely, we propose to use synthesis techniques to obtain a symbolic representation of the learned policy, which can then be debugged manually or automatically using program repair. After the repair step, we use behavior cloning to obtain the policy corresponding to the repaired program, which is then further improved using gradient descent. This process continues until the learned policy satisfies desired constraints. We instantiate MORL for the simple CartPole problem and show that the programmatic representation allows for high-level modifications that in turn lead to improved learning of the policies.

Subfields: Artificial Intelligence, Software

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chao Yu
2. Jiming Liu
3. Shamim Nemati
4. Guoqi Ren

Citations in the corpus

• Reinforcement Learning in Healthcare: A Survey (2019, background): “…of interpretability renders the policies unable to reveal the real correlation between features in the data and specific actions, and to impose and verify certain desirable policy properties, such as worst-case guarantees or safety constraints, for further policy debugging and improvement [21].”