Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces

By Aapo Hyvärinen, Patrik O. Hoyer. 2000

635 citations recorded by the source corpus.

Subfields: Biophysics, Computer Vision and Pattern Recognition, Media Technology, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Computer Vision and Pattern Recognition

In Media Technology

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

Disentangling Factors of Variation via Generative Entangling (2012, background): “…together from purely unsupervised data, and thereby extract directions of variance over which the pooling features become invariant (Kohonen et al., 1979; Ko-honen, 1996; Hyv¨arinen and Hoyer, 2000; Le et al., 2010; Kavukcuoglu et al., 2009; Ranzato and Hinton, 2010; Courville et al., 2011b).”
Group Equivariant Convolutional Networks (2016, methodology): “Then we take a look at functions on groups (used to model feature maps in G-CNNs) and their transformation properties.”
Density Modeling of Images using a Generalized Normalization Transformation (2015, methodology): “This is the Independent Subspace Analysis model (ISA; Hyvärinen & Hoyer, 2000), expressed as a Gaussianizing transformation.”
Deep Learning of Representations: Looking Forward (2013, background): “…associated continuous-valued factors (e.g., pose parameters), as in transforming auto-encoders (Hinton et al., 2011), spike-and-slab RBMs with pooled slab variables (Courville et al., 2011) and other pooling-based models that learn a feature subspace (Kohonen, 1996; Hyvärinen and Hoyer, 2000).”
A Survey of Inductive Biases for Factorial Representation-Learning (2016, methodology): “For example, sparse distribution bias is combined with invariance bias in the ISA model, with multilinear bias in a sparse bilinear model, and with hierarchical layers in R-ICA.”
Learning to Relate Images (2011, methodology): “This approach is known as “Independent Subspace Analysis” (ISA) [23].”
Deep learning with COTS HPC systems (2013, background): “The ﬁrst two layers implement selective features (“simple cells”) and invariant features (Hyv¨arinen & Hoyer, 2000; Hyv¨arinen et al., 2001) (“complex cells” (Hubel & Wiesel, 1959)).”
Fault recognition method for speed-up and speed-down process of rotating machinery based on independent component analysis and Factorial Hidden Markov Model (2006, neutral): “[10–23].”
A Fast Fixed-Point Algorithm for Independent Component Analysis of Complex Valued Signals (2000, neutral): “G2(y) = log(a2 +y) ;g 2 ( y )= 1 a 2 + y (9) G3(y )= 1 2 y 2 ;g 3 ( y )= y ( 10 )”
Bibliography of Self-Organizing Map SOM) Papers: 1998-2001 Addendum (2003, neutral): “software [199, 283, 336, 338, 391, 423, 636, 455, 520, 557, 560, 581, 834, 709, 805, 904, 905, 1110, 1253, 1285, 1355, 1378, 1384, 1449, 1464, 1500, 1509, 1530, 1668, 1672, 1722, 1732, 1795, 1794, 2087, 1833, 1874, 1897, 2032, 2033, 2031] solder joint(s) [840, 1499, 2051] sonar [14, 245, 739, 860, 1”
Independent component analysis: recent advances (2013, background): “In the earliest work, the vi were divided into groups (or subspaces) such that the variables in the same group are positively correlated, while the variables in different groups are independent [39].”
Towards a general independent subspace analysis (2006, methodology): “Hyvärinen and Hoyer presented a special case of k-ISA by combining it with invariant feature subspace analysis [7].”
Nonlinear Extraction of Independent Components of Natural Images Using Radial Gaussianization (2009, background): “One possible solution is to assume that the image density can be partitioned into independent elliptically symmetric subspaces (Hyv¨arinen & Hoyer, 2000; Wainwright, Simoncelli, & Willsky, 2001).”
From learning models of natural image patches to whole image restoration (2011, methodology): “To compensate for this, [2] apply a non-linear decay function over higher values of the block variance histogram in an iterative manner.”
On the Analysis and Interpretation of Inhomogeneous Quadratic Forms as Receptive Fields (2006, methodology): “Quadratic forms were also used in various theoretical papers, either explicitly (Hashimoto, 2003; Bartsch and Obermayer, 2003) or implicitly in the form of neural networks (Hyvärinen and Hoyer, 2000; Hyvärinen and Hoyer, 2001; Körding et al., 2004).”
Independent component analysis applied to feature extraction from colour and stereo images (2000, methodology): “Our main data consists of colour versions of natural scenes (depicting forest, wildlife, rocks, etc.) which we have used in previous work as well (Hyvärinen, 1999b; Hyvärinen and Hoyer, 2000).”
Document clustering using nonnegative matrix factorization (2006, methodology): “This scheme is applicable to the decomposition of datasets into independent feature subspaces by Hyvärinen and Hoyer [7].”
Hierarchical kernel stick-breaking process for multi-task image analysis (2008, methodology): “Hyv¨arinen, A., & Hoyer, P. (2000).”