Tensor Analyzers

By Yichuan Tang, Ruslan Salakhutdinov, Geoffrey E. Hinton. 2013

58 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Towards a Definition of Disentangled Representations (2018, background): “A number of purely unsupervised approaches to disentangled factor learning have been proposed [67, 24, 74], however they did not tend to scale well beyond toy datasets.”
Isolating Sources of Disentanglement in VAEs (2018, background): “Prior to β -VAE [7], some have shown successful disentanglement in limited settings with few factors of variation [1, 14, 30, 31].”
Attend, Infer, Repeat: Fast Scene Understanding with Generative Models (2016, background): “…range from highly speciﬁed (Milch et al., 2005; Mansinghka et al., 2013), to partially speciﬁed (Zhu & Mumford, 2006; Roux et al., 2011; Heess et al., 2011; Eslami & Williams, 2014; Tang et al., 2013; 2014), to largely unspeciﬁed (Hinton, 2002; Salakhutdinov & Hin-ton, 2009; Eslami et al., 2012).”
Understanding disentangling in β-VAE (2018, background): “A number of purely unsupervised approaches to disentangled factor learning have been proposed [36, 10, 39, 8, 9, 6, 15], including β -VAE [15], the focus of this text. β -VAE is a state of the art model for unsupervised visual disentangled representation learning.”
A Survey of Inductive Biases for Factorial Representation-Learning (2016, methodology): “For example, Tensor Analyzers [51] are bilinear models that assume Gaussian-distributed factors, and were applied to model lighting conditions and identity on a face dataset.”
Toward an Integration of Deep Learning and Neuroscience (2016, background): “ parts and their transformations (Wang and Yuille, 2014), in eﬀect performing a rendering of a scene based on its structural description. Hinton and colleagues have also proposed cortical “capsules” (Tang et al., 2013, 2012; Hinton et al., 2011) for compositional inverse rendering. The network can t”
The student-t mixture as a natural image patch prior with application to image compression (2014, background): “Furthermore recent work (Theis et al., 2011; Tang et al., 2013) showed that many of these models, such as restricted Boltzmann machines and deep belief networks are outperformed by more basic models such as the Gaussian Mixture model (GMM) in terms of log likelihood on real-valued data.”
Early Visual Concept Learning with Unsupervised Deep Learning (2016, background): “So far any purely unsupervised approaches to disentangled factor learning have not scaled well [11, 30, 9, 10].”
Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions (2016, unknown): “7 Nonlinear Tensor Decompositions – Infinite Tucker The Infinite Tucker model and its modification, the Distributed Infinite Tucker (DinTucker), generalize the standard Tucker decomposition to infinite feature spaces using the kernel and Bayesian approaches [252, 284, 294].”
Learning to Disentangle Factors of Variation with Manifold Interaction (2014, methodology): “The tensor analyzer (TA) (Tang et al., 2013) extended factor analysis by introducing a factor loading tensor to model the interaction among multiple groups of latent factor units, and was applied to modeling lighting and face morphology.”
Isolating Sources of Disentanglement in Variational Autoencoders (2018, background): “Prior to β-VAE (Higgins et al., 2017), some have shown successful disentanglement in limited settings with few factors of variation (Schmidhuber, 1992; Tang et al., 2013; Desjardins et al., 2012; Glorot et al., 2011).”
Deep Visual Analogy-Making (2015, methodology): “Multiplicative interactions 2For a tensor W ∈ RK×K×K and vectors v, w ∈ RK , we define the tensor multiplication W ×1 v×2 w ∈ RK as (W ×1 v ×2 w)l = ∑K i=1 ∑K j=1 Wijlviwj , ∀l ∈ {1, ...,K}. were similarly used in bilinear models [27], disentangling Boltzmann Machines [23] and Tensor Analyzers [26].”