NerveNet: Learning Structured Policy with Graph Neural Networks

By Tingwu Wang, Renjie Liao, Jimmy Ba, Sanja Fidler. 2018

327 citations recorded by the source corpus.

Subfields: Reinforcement Learning, Graph Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Graph Learning

Citations in the corpus

Structured agents for physical construction (2019, neutral): “These models have also begun to be incorporated into model-free and model-based RL, in domains such as combinatorial optimization, motor control, and game playing (Dai et al., 2017; Kool & Welling, 2018; Wang et al., 2018; Sanchez-Gonzalez et al., 2018; Zambaldi et al., 2019).”
MONet: Unsupervised Scene Decomposition and Representation (2019, background): “…networks that rely on hand-crafted object representations have recently achieved remarkable results in a wide range of research areas, including reinforcement learning, physical modeling, and multi-agent control [Battaglia et al., 2018, Wang et al., 2018, Hamrick et al., 2017, Hoshen, 2017].”
Relational inductive biases, deep learning, and graph networks (2018, methodology): “…et al., 2015; Gilmer et al., 2017), to predict traﬃc on roads (Li et al., 2017; Cui et al., 2018), to classify and segment images and videos (Wang et al., 2018c; Hu et al., 2017) and 3D meshes and point clouds (Wang et al., 2018d), to classify regions in images (Chen et al., 2018a), to…”
Deep Multi-Agent Reinforcement Learning with Relevance Graphs (2018, methodology): “Furthermore, we propose the use of message generation techniques from this graph, inspired by the NerveNet architecture [18].”
Graph networks as learnable physics engines for inference and control (2018, methodology): “Our approach may also be useful within imagination-based planning frameworks (Hamrick et al., 2017; Pascanu et al., 2017), as well as integrated architectures with GN-like policies (Wang et al., 2018).”
Can Wikipedia Help Offline Reinforcement Learning? (2022, background): “…autoregressive language modeling on a relatively large dataset, showing promising initial success not only on its ability to scale to large models sizes, but also for its impressive performance when ﬁne-tuning on task-speciﬁc natural language understanding (NLU; Wang et al., 2018a) datasets.”
Neural Logic Reinforcement Learning (2019, methodology): “he abstract concepts as entities and relationships between them and perform deduction on the relations. The graph-based relational inductive bias has been tested in RL context (Zambaldi et al., 2018; Wang et al., 2018) and showed signiﬁcantly better generalization compared with Multilayer Perceptron”
Neural Task Graphs: Generalizing to Unseen Tasks From a Single Video Demonstration (2018, methodology): “The use of compositionality has led to better generalization in Visual Question Answering [17, 20, 24] and Policy Learning [3, 7, 38].”
Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning (2019, methodology): “Graph Neural Networks (GNNs) [36], [37], [38], [39], [40], [41], [20], [21].”
Deep Multi-Agent Reinforcement Learning for Decentralized Continuous Cooperative Control (2020, background): “On Multi-Agent Mujoco, COMIX outperforms MADDPG in absolute terms on 2-Agent Walker scenario (Figure 4c), while MADDPG cannot learn it effectively.”
LanczosNet: Multi-Scale Deep Graph Convolutional Networks (2019, methodology): “Ns, including community detection [41], situation recognition [42], RGBD semantic segmentation [43], few-shot learning [21], probabilistic inference [44], continuous control of reinforcement learning [45,46] and so on. Graph based Manifold Learning The non-linear dimensionality reduction methods, su”
Learning to schedule job-shop problems: representation and policy learning using graph neural network and reinforcement learning (2021, background): “As a result, GNN models can successfully perform their tasks without additional training in various fields including robot control, wind power estimation, and city-scale weather prediction.[32, 48, 39].”
Deep Reinforcement Learning (2018, background): “ng (NLP).Xu et al.(2015) integrate attention to image captioning. See alsoLiu et al.(2016),Lu et al.(2016),Rennie et al. (2017), andRen et al.(2017) for image captioning. SeePasunuru and Bansal(2017);Wang et al. (2018f) for video captioning.Strub et al.(2017) propose end-to-end optimization with dee”
MAGNet: Multi-agent Graph Network for Deep Multi-agent Reinforcement Learning (2019, methodology): “The agent AI responsible for decisionmaking is also represented as a neural network whose inputs are accumulated messages (generated by a method inspired by NerveNet [18] and described below) and the current state of the environment.”
DOM-Q-NET: Grounded RL on Structured Language (2019, background): “We also propose to use three separate multilayer perceptrons (MLP) (Rumelhart et al., 1985) to parametrize a factorized Q function for different action categories: “click”, “type” and “mode”.”
Know Your Action Set: Learning Action Relations for Reinforcement Learning (2022, methodology): “Graph neural networks (Battaglia et al., 2018) have been explored in RL tasks with a rich relational structure, such as morphological control (Wang et al., 2018; Sanchez-Gonzalez et al., 2018; Pathak et al., 2019), multi-agent RL (Tacchetti et al., 2019), physical construction (Hamrick et al.,…”
Graph Matching Networks for Learning the Similarity of Graph Structured Objects (2019, methodology): “Most of the previous work on GNNs focus on supervised prediction problems (with exceptions like (Dai et al., 2017; Li et al., 2018; Wang et al., 2018a)).”
Graph Element Networks: adaptive, structured computation and memory (2019, background): “Non-local neural networks (Wang et al., 2018b) also use GNNs to model long-range interactions in video without a traditional notion of entity, but the nodes are tied to a CNN channel and the graph is required to be complete, which limits the applicability of the model.”