Learning to Represent Spatial Transformations with Factored Higher-Order Boltzmann Machines

By Roland Memisevic, Geoffrey E. Hinton. 2010

287 citations recorded by the source corpus.

Subfields: Biophysics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Probabilistic & Bayesian Methods

Citations in the corpus

Predicting Parameters in Deep Learning (2013, background): “ no reason we could not incorporate deeper columns into our networks, and this would make for a potentially interesting avenue of future work. Our approach is superﬁcially similar to the factored RBM [22, 20, 25], whose parameters form a 3-tensor. Since the total number of parameters in this model i”
Representation Learning: A Review and New Perspectives (2012, methodology): “It has been used in a truly impressive variety of applications, including fMRI image classification [180], motion and spatial transformations [201], [144], collaborative filtering [178], and natural image modeling [160], [53].”
Learning to See by Moving (2015, background): “The aim of such approaches has been to discover compact and rich representations of images that are also sufﬁcient to reconstruct these images.”
Video (language) modeling: a baseline for generative models of natural videos (2014, methodology): “Bilinear models of transformations between nearby frames have also been investigated by Memisevic & Hinton (2009); Sutskever et al. (2009); Taylor et al. (2011); Michalski et al. (2014) as well as Miao & Rao (2007) via Lie group theory.”
Deep learning in neural networks: An overview (2014, background): “The Deep Belief Network (DBN) is a stack of Restricted Boltzmann Machines (RBMs) (Smolensky, 1986), which in turn are Boltzmann Machines (BMs) (Hinton & Sejnowski, 1986) with a single layer of feature-detecting units; compare also Higher-Order BMs (Memisevic & Hinton, 2010).”
Disentangling Factors of Variation via Generative Entangling (2012, methodology): “The proposed model is also closely re-lated to recent work (Memisevic and Hinton, 2010), where higher-order Boltzmann Machines are used as models of spatial transformations in images.”
Harmonic Networks: Deep Translation and Rotation Equivariance (2016, background): “[25] construct a higher-order Boltzmann machine, which learns tuples of transformed linear filters in input–output pairs.”
Learning the Irreducible Representations of Commutative Lie Groups (2014, neutral): “Other, non-group-theoretical approaches to learning transformations and invariant representations exist (Memisevic & Hinton, 2010).”
DeepMPC: Learning Deep Latent Features for Model Predictive Control (2015, background): “We do so by introducting factored conditional units [27], where features from some number of inputs modulate multiplicatively”
Discovering Hidden Factors of Variation in Deep Networks (2014, methodology): “motion [4, 5, 6, 7, 8, 9], have relied upon a bilinear model architecture in which the units representing different factors are combined multiplicatively.”
Exponentially Increasing the Capacity-to-Computation Ratio for Conditional Computation in Deep Learning (2014, positive): “One can view the above as a generalization of three-way connections found in some models (Memisevic and Hinton, 2010; Sutskever et al. , 2011) to k + 2 -way interactions (between the k gating bits, the input x j and each output h i ).”
Deep learning for detecting robotic grasps (2013, background): “corporate new input modalities such as RGB-D cameras. Recent methods based on deep learning [1] have demonstrated state of the art performance in a wide variety of tasks, including visual recognition [21, 46, 27, 43, 20], audio recognition [25, 11], and natural language processing [5, 6]. These tech”
Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning (2017, methodology): “…For combining condition variables into a neural network (e.g., combining task embedding into the convolutional network in the parameterized skill), we used a form of multiplicative interaction instead of concatenating such variables as suggested by (Memisevic and Hinton, 2010; Oh et al., 2015).”
Learning to Relate Images (2011, methodology): “The vector-ﬁeld was produced by (i) inferring the transformation given the image pair (Eq.”
Preliminary investigation of Boltzmann machine classifiers for speaker recognition (2012, methodology): “(The variational Bayes EM training algorithm extends to straightforwardly to third order Boltzmann machines although steps need to be taken to control the number of free parameters to be estimated [17].”
Bootstrapping bilinear models of Simple Vehicles (2015, methodology): “The computation (15) is similar to three-way Hebbian learning between y, ẏ, and u.”