Probabilistic Symmetries and Invariant Neural Networks

By Benjamin Bloem-Reddy, Yee Whye Teh. 2020

191 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

BayesFlow: Learning Complex Stochastic Models With Invertible Neural Networks (2020, methodology): “Recent work on probabilistic symmetry [5] and algorithmic alignment [57] as well as our current experiments do, however, provide some insights about the design of summary networks.”
Relational Pooling for Graph Representations (2019, methodology): “In parallel, Bloem-Reddy & Teh (2019) generalizes functions that are invariant to graph permutations to those that are invariant to general group actions, discussing the connections to exchangeable probability distributions (De Finetti, 1937; Aldous, 1981; Diaconis & Janson, 2008).”
Conditional Set Generation with Transformers (2020, background): “Permutation equivariance imposes a constraint on the structure of the model (Bloem-Reddy & Teh, 2018; 2019), and therefore sets are often treated as an ordered collection of items, which allows using standard machine learning models.”
Lossy Compression for Lossless Prediction (2021, background): “4 in Eaton [19], Theorem 7 in Bloem-Reddy and Teh [52]), to any equivalences (rather than only group orbits) and without making the assumption of (marginal) invariance of P (X) to ∼.”
Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs (2018, background): “More broadly, there are other connections between permutation-invariant deterministic functions and exchangeability in probability distributions, as recently discussed by Bloem-Reddy & Teh (2019).”
Universal approximations of permutation invariant/equivariant functions by deep neural networks (2019, background): “Many researchers studied invariant/equivariant networks as [14], [23], [21], [4] and [17].”
Delving Deeper into Anti-Aliasing in ConvNets (2020, background): “Anti-aliasing is also highly related to geometric transformation invariance, which is explored in several recent works [77,33,5,54].”
On the equivalence between graph isomorphism testing and function approximation with GNNs (2019, background): “The article [4] provides a nice and general theoretical framework that establishes an equivalence between the functional and probabilistic perspectives to symmetry via noise outsourcing in both general and particular settings.”
Learning Robust Representations via Multi-View Information Bottleneck (2020, background): “, 2017; Chen et al., 2017). The idea of discarding irrelevant information was introduced in Tishby et al. (2000) and identified as one of the possible reasons behind the generalization capabilities of deep neural networks by Tishby & Zaslavsky (2015) and Achille & Soatto (2018).”
Automatic Posterior Transformation for Likelihood-Free Inference (2019, background): “Exchangeable neural networks (Za-heer et al., 2017; Chan et al., 2018; Bloem-Reddy & Teh, 2019) could also be applied when x o consists of multiple i.i.d. observations from the same parameters.”