Transfer Reinforcement Learning with Shared Dynamics

By Romain Laroche, M. Barlier. 2017

60 citations recorded by the source corpus.

This article addresses a particular Transfer Reinforcement Learning (RL) problem: when dynamics do not change from one task to another, and only the reward function does. Our method relies on two ideas, the first one is that transition samples obtained from a task can be reused to learn on any other task: an immediate reward estimator is learnt in a supervised fashion and for each sample, the reward entry is changed by its reward estimate. The second idea consists in adopting the optimism in the face of uncertainty principle and to use upper bound reward estimates. Our method is tested on a navigation task, under four Transfer RL experimental settings: with a known reward function, with strong and weak expert knowledge on the reward function, and with a completely unknown reward function. It is also evaluated in a Multi-Task RL experiment and compared with the state-of-the-art algorithms. Results reveal that this method constitutes a major improvement for transfer/multi-task problems that share dynamics.

Subfields: Artificial Intelligence, Computer Networks and Communications, Computer Science Applications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Computer Science Applications

Citations in the corpus

Safe Policy Improvement with Baseline Bootstrapping (2017, methodology): “[15] Romain Laroche and Merwan Barlier.”
Attentive Multi-Task Deep Reinforcement Learning (2019, methodology): “i-task reinforcement learning. We use a neural network based attention mechanism to focus on sub-networks depending on the current state of the environment and the task to be solved. Most recent work [29,12,15,2] in the multi-task/transfer deep reinforcement learning setting capitalize on some share”
Transforming task representations to perform novel tasks (2020, background): “Work in model-based RL has partly addressed how to transfer knowledge between different reward functions (35).”
NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning (2021, methodology): “This protocol implicitly requires a transferable environment which contradicts practical offline RL setting, since we can learn from these domains and adapt to the online environment as done in [29, 30].”
Context-Aware Policy Reuse (2018, background): “[20] reuses transition samples obtained from one task to accelerate learning of another, but it is constrained with the assumption that transition samples can be shared across tasks.”