COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration

By Nicholas Watters, Loic Matthey, Matko Bosnjak, Christopher P. Burgess, Alexander Lerchner. 2019

133 citations recorded by the source corpus.

Data efficiency and robustness to task-irrelevant perturbations are long-standing challenges for deep reinforcement learning algorithms. Here we introduce a modular approach to addressing these challenges in a continuous control environment, without using hand-crafted or supervised information. Our Curious Object-Based seaRch Agent (COBRA) uses task-free intrinsically motivated exploration and unsupervised learning to build object-based models of its environment and action space. Subsequently, it can learn a variety of tasks through model-based search in very few steps and excel on structured hold-out tests of policy robustness.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Object-Centric Learning with Slot Attention (2020, background): “[65] Nicholas Watters, Loic Matthey, Matko Bosnjak, Christopher P Burgess, and Alexander Lerchner.”
Toward Causal Representation Learning (2021, background): “Structured generative approaches further aim at decomposing an environment into multiple entities with causally correct relations among them, modulo the completeness of the variables, and confounding [15], [44], [59], [136], [264], [265].”
Towards Causal Representation Learning (2021, background): “among them, modulo the completeness of the variables, and confounding [58, 265, 43, 264, 14, 136].”
Unsupervised Model Selection for Variational Disentangled Representation Learning (2019, methodology): “…model ranking produced by UDR correlates with task performance on two different domains: the fairness on a classiﬁcation task introduced by Locatello et al. (2018), and data efﬁciency on a clustering task for a model-based reinforcement learning agent introduced by Watters et al. (2019) (see Sec.”
Hamiltonian Generative Networks (2019, background): “This is true for intelligence in both biological (Friston, 2009; 2010; Clark, 2013) and artiﬁcial (Hafner et al., 2019; Battaglia et al., 2013; Watter et al., 2015; Watters et al., 2019) systems.”
R-SQAIR: Relational Sequential Attend, Infer, Repeat (2019, background): “Its learned generative model is potentially useful as part of a world simulator [24, 23, 13, 33].”
Contrastive Learning of Structured World Models (2019, methodology): “Several recent works succeed in learning such structured models directly from pixels (Watters et al., 2017; van Steenkiste et al., 2018; Xu et al., 2019; Watters et al., 2019), but in contrast to our work rely on pixel-based loss functions.”
Hierarchical Relational Inference (2020, background): “Typically, these approaches assume access to supervised state descriptions, and only few works also infer object representations from raw visual images (van Steenkiste et al. 2018; Veerapaneni et al. 2019; Watters et al. 2019).”
Entity Abstraction in Visual Model-Based Reinforcement Learning (2019, result): “COBRA uses MONet [46] to obtain entity representations upon which the authors train an exploration policy based on the prediction errors of the dynamics model.”