CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents

By Sam Powers, Eliot Xing, Eric Kolve, Roozbeh Mottaghi, Abhinav Kumar Gupta. 2021

50 citations recorded by the source corpus.

Progress in continual reinforcement learning has been limited due to several barriers to entry: missing code, high compute requirements, and a lack of suitable benchmarks. In this work, we present CORA, a platform for Continual Reinforcement Learning Agents that provides benchmarks, baselines, and metrics in a single code package. The benchmarks we provide are designed to evaluate different aspects of the continual RL challenge, such as catastrophic forgetting, plasticity, ability to generalize, and sample-efficient learning. Three of the benchmarks utilize video game environments (Atari, Procgen, NetHack). The fourth benchmark, CHORES, consists of four different task sequences in a visually realistic home simulator, drawn from a diverse set of task and scene parameters. To compare continual RL methods on these benchmarks, we prepare three metrics in CORA: Continual Evaluation, Isolated Forgetting, and Zero-Shot Forward Transfer. Finally, CORA includes a set of performant, open-source baselines of existing algorithms for researchers to use and expand on. We release CORA and hope that the continual RL community can benefit from our contributions, to accelerate the development of new continual RL algorithms.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Dynamic Dialogue Policy for Continual Reinforcement Learning (2022, methodology): “…an implementation of the state-of-the-art continual RL algorithm (Rolnick et al., 2018) and continual learning metrics for evaluation based on Powers et al. (2021), and • a further, more sophisticated, new continual learning architecture based on the transformer encoder-decoder (Vaswani et…”
Building a Subspace of Policies for Scalable Continual Learning (2022, methodology): “While current methods perform well along some of these dimensions, they tend to suffer along others [32, 49].”
Dynamic Dialogue Policy Transformer for Continual Reinforcement Learning (2022, methodology): “Similar to Powers et al. (2021), we average the metrics F i,j and F i,l if j and l refer to the same task z .”
Online Continual Learning For Interactive Instruction Following Agents (2024, methodology): “Here, the tasks in our proposed continual learning setups are similar to (Powers et al., 2022) that simplifies the task setup of (Shridhar et al., 2020).”
LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning (2023, unknown): “ContinualWorld [69] modiﬁes the 50 manipulation tasks in MetaWorld for LL. CORA [51] builds four lifelong RL benchmarks based on Atari, Procgen [15], MiniHack [58], and ALFRED [62].”
L2Explorer: A Lifelong Reinforcement Learning Assessment Environment (2022, methodology): “Finally, the critical component is linking this procedurally generated environment to careful experimental design and metrics, similar to CoRA (Powers et al. 2021).”