Natural image statistics and efficient coding.

By Barbara Olshausen, David J. Field. 1996

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Measuring Data (2022, background): “…many approaches that aim to approximate the probability density functions (PDFs) of images using approaches ranging from modeling the low-level statistics of natural images (Olshausen and Field, 1996) to more complex approaches such as Variational Auto-Encoders (VAEs) (Kingma and Welling, 2013).”
Sparse coding with an overcomplete basis set: a strategy employed by V1? (1997, methodology): “It was shown previously that when such a code is sought for natural images, the basis functions that emerge are qualitatively similar in form to simple cell receptive fields and also to the basis functions of certain wavelet transforms (Olshausen & Field, 1996a).”
Why does natural scene categorization require little attention? Exploring attentional requirements for natural and synthetic stimuli (2005, background): “One might argue that in the above manipulation, even though the tested natural scene category was not trained, it was nevertheless learned during training because natural scene photographs shared much commonalities (Olshausen & Field, 1996).”
K-SVD and its non-negative variant for dictionary design (2005, background): “Subsequent work by Lewicki, Olshausen and Sejnowski, proposed several direct extensions of this work [10].”
Sparse Coding Of Time-Varying Natural Images (2010, background): “In earlier work (Olshausen & Field, 1996; 1997), we have shown that the higher-order forms of structure in static images may be characterized in part by a linear generative model that attempts to describe images in terms of sparse, statistically independent components.”
Understanding the (un)interpretability of natural image distributions using generative models (2019, background): “For example, [18] models the low-level statistics of natural images.”
Gaussian Kullback-Leibler approximate inference (2013, background): “Natural images for instance are known to have sparse statistics over a range of linear filters (an example filter being the difference in intensities of neighbouring pixels) (Olshausen and Field, 1996).”
Population coding under normalization (2009, background): “…dependencies of neural activity, and have put forward the idea that normalization may serve to optimize the representation of natural signals (Fairhall, Lewen, Bialek, & de Ruyter Van Steveninck, 2001; Olshausen & Field, 1996a, 1996b, 2004; Ruderman & Bialek, 1994; Schwartz & Simoncelli, 2001).”
The "independent components" of natural scenes are edge filters. (1997, unknown): “FIGURE 1. The Blind Linear hnage Synthesis model ( Olshausen & Field, 1996 ).”
The Nonlinear Statistics of High-Contrast Patches in Natural Images (2003, background): “projection pursuit (Huber, 1985; Friedman, 1987), sparse coding (Olshausen and Field, 1996) and ICA (see Hyvärinen (1999) for a survey of ICA and related methods).”
A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients (2000, background): “Especially relevant are recent results on choosing bases to optimize information-theoretic criterion, which suggest that a basis of localized oriented operators at multiple scales is optimal for image representation [43, 2].”
The Intrinsic Dimension of Images and Its Impact on Learning (2021, methodology): “This observation motivated work on efﬁcient coding (Olshausen & Field, 1996) and served as a prior in computer vision (Peyr´e, 2009).”