Permutation-equivariant neural networks applied to dynamics prediction
By Nicholas Guttenberg, Nathaniel Virgo, Olaf Witkowski, Hidetoshi Aoki, Ryota Kanai. 2016
61 citations recorded by the source corpus.
The introduction of convolutional layers greatly advanced the performance of neural networks on image tasks due to innately capturing a way of encoding and learning translation-invariant operations, matching one of the underlying symmetries of the image domain. In comparison, there are a number of problems in which there are a number of different inputs which are all 'of the same type' --- multiple particles, multiple agents, multiple stock prices, etc. The corresponding symmetry to this is permutation symmetry, in that the algorithm should not depend on the specific ordering of the input data. We discuss a permutation-invariant neural network layer in analogy to convolutional layers, and show the ability of this architecture to learn to predict the motion of a variable number of interacting hard discs in 2D. In the same way that convolutional layers can generalize to different image sizes, the permutation layer we describe generalizes to different numbers of objects.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Günter Klambauer
- Lukas Gruber
- Hubert Ramsauer
- Sepp Hochreiter
- Johannes Brandstetter
- Markus Holzleitner
- Michael Widrich
- Philipp Seidl
Citations in the corpus
- UvA-DARE (Digital Academic Repository) Neural Relational Inference for Interacting Systems Neural Relational Inference for Interacting Systems (2018, unknown): “Several recent works have studied the problem of learning the dynamics of a physical system from simulated trajectories (Battaglia et al., 2016; Guttenberg et al., 2016; Chang et al., 2017) and from generated video data (Watters et al., 2017; van Steenkiste et al., 2018) with a graph neural network.”
- Hopfield Networks is All You Need (2020, methodology): “Possible applications for a Hopfield layer in a deep network architecture comprise: • multiple instance learning (MIL) [26], • processing of and learning with point sets [51, 52, 73], • set-based and permutation invariant learning [32, 55, 76, 40, 38, 78], • attention-based learning [64], • associat”