Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer Network 5

By Jean-Pierre Nadal, N. Parga. 1994

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “s (Bayer et al., 2013). Dropout is closely related to older, biologically plausible techniques for adding noise to neurons or synapses during training (e.g., Murray and Edwards, 1993; Schuster, 1992; Nadal and Parga, 1994; Jim et al., 1995; An, 1996), which in turn are closely related to ﬁnding pert”
A Unifying Information-Theoretic Framework for Independent Component Analysis (2000, background): “3 Information Maximization Nadal and Parga (1994) showed that in the low-noise case, the maximum of the mutual information between the inputs x and outputs y of a neural processor implied that the output distributions were factorial.”
Face recognition by independent component analysis (2002, background): “of the nonlinear transfer function is the same as the cumulative density functions of the underlying ICs (up to scaling and translation) it can be shown that maximizing the joint entropy of the outputs in also minimizes the mutual information between the individual outputs in [12], [42].”
Independent component analysis: algorithms and applications (2000, background): “Another related contrast function was derived from a neural network viewpoint in (Bell and Sejnowski, 1995; Nadal and Parga, 1994 ).”
Independent component analysis: recent advances (2013, result): “Furthermore, a neural network approach called infomax was proposed by Bell and Sejnowski (1995); Nadal and Parga (1994) and was shown to be equivalent to likelihood by Cardoso (1997).”
Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information (1997, methodology): “It is related to the redundancy reduction principle (Barlow & Földiák, 1989; Nadal & Parga, 1994), one of the principles possibly employed by the nervous system to extract the statistically independent features for a better representation of the input without losing information.”
Natural Gradient Works Efficiently in Learning (1998, background): “The expected loss is which represents the entropy of the output y after a componentwise non-linear transformation (Nadal & Parga, 1994; Bell & Sejnowski, 1995).”
Removing electroencephalographic artifacts by blind source separation. (2000, unknown): “…1996; Cichocki, Unbehauen, & Rummert, 1994; Comon, 1994; Girolami & Fyfe, 1997; Karhunen, Oja, Wang, Vigário, & Joutsensalo, 1996; Lambert, 1996; Nadal & Parga, 1994; Pearlmutter & Parra, 1997; Pham, 1997; Roth & Baram, 1996; Yellin & Weinstein, 1996!. Comon~1994! defined the concept of ICA as…”
The "independent components" of natural scenes are edge filters. (1997, unknown): “…is the same (up to scaling and shifting) as the cumulative density functions (c.d.f.s) of the underlying independent components, it can be shown (Nadal & Parga, 1994)* that such a nonlinear "infomax" procedure also minimizes the mutual information between the ui, exactly what is required for…”