Feature Extraction Through LOCOCODE

By Sepp Hochreiter, J. Urgen Schmidhuber. 1999

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) (2015, background): “Therefore the representation is both noise-robust and low-complex (Hochreiter & Schmidhuber, 1999).”
Deep learning in neural networks: An overview (2014, methodology): “Flat Minimum Search (FMS) (Hochreiter & Schmidhuber, 1997a, 1999) searches for a ‘‘flat’’ minimum of the error function: a large connected region in weight space where error is low and remains approximately constant, that is, few bits of information are required to describe low-precision weights wit”
Weakly-Supervised Disentanglement Without Compromises (2020, methodology): “Relation to prior work Closely related to the proposed objective (6) the GVAE of Hosoya (2019) and the ML-VAE of Bouchacourt et al.”
Are Disentangled Representations Helpful for Abstract Visual Reasoning? (2019, background): “Other Related Works Learning disentangled representations is similar in spirit to non-linear ICA, although it relies primarily on (architectural) inductive biases and different degrees of supervision [13, 2, 39, 36, 37, 38, 25, 33, 32].”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, background): “2 M ’s Training To decrease bitsM + bitsH , we add a regularizing term to E, to punish excessive complexity [4, 5, 91, 294, 155, 135, 97, 170, 169, 104, 290, 7, 290, 87, 286, 319, 100, 102].”
Neural Networks: Tricks of the Trade (2012, methodology): “Neural Computation 4, 448–472 (1992) [9] Neal, R.”
Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction (2011, methodology): “Methods using this paradigm include stacks of: Low-Complexity Coding and Decoding machines (LOCOCODE) [10], Predictability Minimization layers [23] [24], Restricted Boltzmann Machines (RBMs) [8], auto-encoders [20] and energy based models [15].”
Unsupervised Learning in LSTM Recurrent Neural Networks (2001, background): “The few exceptions [30,34‐39] include (a) toy applications of predictability minimization [7,40] to the problem of encoding input sequences through RNNs in a nonredundant fashion [35,36], (b) unsupervised RNN-based compression of input sequences to facilitate subsequent processing by another RNN [37”
Nonlinear ICA through low-complexity autoencoders (1999, methodology): “As a preprocessor for a vowel recognition benchmark problem it sets the stage for excellent classification performance [10].”
Meta-Learning Update Rules for Unsupervised Representation Learning (2018, methodology): “Other techniques such as Schmidhuber (1992), Hochreiter and Schmidhuber (1999), and Olshausen and Field (1997) deﬁne various desirable properties about the latent representation of the input, such as predictability, complexity of encoding mapping, independence, or sparsity, and optimize to achieve…”
Advances in Nonlinear Blind Source Separation (2003, methodology): “Anyway, the experimental results yielded by the LOCOCODE method show interesting connections with ICA; sometimes the method provides a nonlinear ICA solution, sometimes it does not [ 92 ].”