Deep reinforcement learning with relational inductive biases

By Vinícius Flores Zambaldi, David Raposo, Adam Santoro, Victor Bapst, Yujia Li, Igor Babuschkin, Karl Tuyls, David P. Reichert, Timothy P. Lillicrap, Edward Lockhart and 6 more. 2018

231 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Model Architectures

Citations in the corpus

Rethinking Attention with Performers (2020, background): “…Transformer scope as a more scalable replacement for regular attention, which itself has a wide variety of uses in computer vision (Fu et al., 2019), reinforcement learning (Zambaldi et al., 2019), training with softmax cross entropy loss, and even combinatorial optimization (Vinyals et al., 2015).”
Attention Augmented Convolutional Networks (2019, background): “Using self-attention in cooperation with convolutions is a theme shared by recent work in Natural Language Processing [49] and Reinforcement Learning [52].”
Toward Causal Representation Learning (2021, background): “Machine learning models that incorporate or learn structural knowledge of an environment have been shown to be more efﬁcient and generalize better [9], [11], [15], [16], [27], [58], [77], [84], [85], [141], [157], [177], [181], [197], [211], [212], [244], [258], [272], [274].”
Directional Message Passing for Molecular Graphs (2020, background): “General GNNs have been largely inspired by their application to molecular graphs and have started to achieve breakthrough performance in various tasks at around the same time the molecular variants did (Kipf & Welling, 2017; Gasteiger et al., 2019; Zambaldi et al., 2019).”
On the Binding Problem in Artificial Neural Networks (2020, methodology): “…heads (i.e. as in Vaswani et al., 2017) may help mitigate this issue and has been successfully applied for relational reasoning about objects (Zambaldi et al., 2019; van Steenkiste et al., 2020; Goyal et al., 2019; Santoro et al., 2018a), citation networks (Veličković et al., 2018), question…”
Structured agents for physical construction (2019, neutral): “These models have also begun to be incorporated into model-free and model-based RL, in domains such as combinatorial optimization, motor control, and game playing (Dai et al., 2017; Kool & Welling, 2018; Wang et al., 2018; Sanchez-Gonzalez et al., 2018; Zambaldi et al., 2019).”
Fitted Q-Learning for Relational Domains (2020, background): “Recently, Zambaldi et al. (2019) proposed to use attention mechanism to capture the interactions between the various entities in the domain.”
Action Semantics Network: Considering the Effects of Actions in Multiagent Systems (2019, background): “Recently, Zambaldi et al. (2019) proposed the relational deep RL to learn environmental entities relations.”
Stabilizing Transformers for Reinforcement Learning (2019, background): “Other work has examined self-attention in the context of exploiting relational structure within the input-space (Zambaldi et al., 2019) or within recurrent memories (Santoro et al., 2018).”
Better Set Representations For Relational Reasoning (2020, methodology): “Typical differentiable methods directly map the input to latent features using a feedforward neural network and partition the latent features into a set representation for downstream reasoning [1, 10, 15, 22].”
Know Your Action Set: Learning Action Relations for Reinforcement Learning (2022, methodology): “…with a rich relational structure, such as morphological control (Wang et al., 2018; Sanchez-Gonzalez et al., 2018; Pathak et al., 2019), multi-agent RL (Tacchetti et al., 2019), physical construction (Hamrick et al., 2018), and structured perception in games like StarCraft (Zambaldi et al., 2018).”
Learning Multi-Agent Communication through Structured Attentive Reasoning (2020, methodology): “The relational communication of our framework is based on the paradigm of relations in agent-based reinforcement learning, which was proposed by [11] through multi-headed dot-product attention (MHDPA) [9].”
Automated curricula through setter-solver interactions (2019, background): “ation is not our setter algorithm, but rather the limitations of the solver agent, for example, the fact that it lacks features like planning (Racaniere et al., 2017) or` relational inductive biases (Zambaldi et al., 2019) that have proven useful for similar tasks. In more complex settings the sette”
A Perspective on Objects and Systematic Generalization in Model-Based RL (2019, methodology): “RL methods that leverage the combinatorics of objects and relations have shown similar beneﬁts in terms of systematic generalization (Zambaldi et al., 2019), sample efﬁ-ciency (Diuk et al., 2008), and transferring skills and knowledge across domains (Kansky et al., 2017).”
Hierarchical Relational Inference (2020, background): “Indeed, several others have proposed to learn object representations in this way (Santoro et al. 2017; Zambaldi et al. 2019).”