Benchmark Environments for Multitask Learning in Continuous Domains

By Peter Henderson, Wei-Di Chang, Florian Shkurti, Johanna Hansen, David Meger, G. Dudek. 2017

42 citations recorded by the source corpus.

As demand drives systems to generalize to various domains and problems, the study of multitask, transfer and lifelong learning has become an increasingly important pursuit. In discrete domains, performance on the Atari game suite has emerged as the de facto benchmark for assessing multitask learning. However, in continuous domains there is a lack of agreement on standard multitask evaluation environments which makes it difficult to compare different approaches fairly. In this work, we describe a benchmark set of tasks that we have developed in an extendable framework based on OpenAI Gym. We run a simple baseline using Trust Region Policy Optimization and release the framework publicly to be expanded and used for the systematic comparison of multitask, transfer, and lifelong learning in continuous domains.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

In Evaluation & Benchmarks

Citations in the corpus

CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning (2020, methodology): “RL methods have been shown to be sensitive to a range of different factors (Henderson et al., 2017b).”
Policy Consolidation for Continual Reinforcement Learning (2019, methodology): “Three pairs of continuous control tasks were used (taken from (Brockman et al., 2016; Henderson et al., 2017; Schulman et al., 2017)); the tasks in each pair were similar to one another, but different enough to induce forgetting in the baseline agents in most cases.”
Continuous Coordination As a Realistic Scenario for Lifelong Learning (2021, background): “, 2016) adding synthetic non-stationarity to the environments (Henderson et al., 2017; Al-Shedivat et al., 2017) or ordering some completely unrelated environments in a sequence (Xu et al.”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “, gravity) to create multiple related tasks (Henderson et al., 2017).”
Knowledge Transfer in Multi-Task Deep Reinforcement Learning for Continuous Control (2020, methodology): “We employed two typical benchmarks (which will be called Benchmarks A and B in the following): 1) Benchmark A: it is called the HalfCheetah task group [7], which includes 8 similar tasks; 2) Benchmark B: it consists of 6 considerably different tasks.”
OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learning (2017, methodology): “We use the HopperSimpleWall-v0 environment from the gym-extensions framework (Henderson et al. 2017) and the RoboschoolHumanoidFlagrun-v1 environment (Schulman et al. 2017).”
An Introduction to Deep Reinforcement Learning (2018, unknown): “To this end, several works have investigated proper metrics and experimental practices when comparing deep RL algorithms (Henderson et al., 2017b; Islam et al., 2017; Machado et al., 2017b; Whiteson et al., 2011).”