Learning Novel Policies For Tasks

By Yunbo Zhang, Wenhao Yu, Greg Turk. 2019

35 citations recorded by the source corpus.

In this work, we present a reinforcement learning algorithm that can find a variety of policies (novel policies) for a task that is given by a task reward function. Our method does this by creating a second reward function that recognizes previously seen state sequences and rewards those by novelty, which is measured using autoencoders that have been trained on state sequences from previously discovered policies. We present a two-objective update technique for policy gradient algorithms in which each update of the policy is a compromise between improving the task reward and improving the novelty reward. Using this method, we end up with a collection of policies that solves a given task as well as carrying out action sequences that are distinct from one another. We demonstrate this method on maze navigation tasks, a reaching task for a simulated robot arm, and a locomotion task for a hopper. We also demonstrate the effectiveness of our approach on deceptive tasks in which policy gradient methods often get stuck.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning explanations that are hard to vary (2020, background): “Looking at agreement to characterize and improve the representations learned by neural networks is an idea that has been explored in recent work, mostly looking at the cosine similarity between gradients [Du et al., 2018, Eshratifar et al., 2018, Fort et al., 2019, Zhang et al., 2019b].”