The Nonlinear Statistics of High-Contrast Patches in Natural Images

By Ann B. Lee, Kim Steenstrup Pedersen, David Mumford. International Journal of Computer Vision, 2003

299 citations recorded by the source corpus.

Recently, there has been a great deal of interest in modeling the non-Gaussian structures of natural images. However, despite the many advances in the direction of sparse coding and multi-resolution analysis, the full probability distribution of pixel values in a neighborhood has not yet been described. In this study, we explore the space of data points representing the values of 3 × 3 high-contrast patches from optical and 3D range images. We find that the distribution of data is extremely “sparse” with the majority of the data points concentrated in clusters and non-linear low-dimensional manifolds. Furthermore, a detailed study of probability densities allows us to systematically distinguish between images of different modalities (optical versus range), which otherwise display similar marginal distributions. Our work indicates the importance of studying the full probability distribution of natural images, not just marginals, and the need to understand the intrinsic dimensionality and nature of the data. We believe that object-like structures in the world and the sensor properties of the probing device generate observations that are concentrated along predictable shapes in state space. Our study of natural image statistics accounts for local geometries (such as edges) in natural scenes, but does not impose such strong assumptions on the data as independent components or sparse coding by linear change of bases.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a11d6a90126d97d61d85098c8731c6f6d781e5ca

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Patch Complexity, Finite Pixel Correlations and Optimal Denoising (2012, background): “Characterizing the properties of natural images is critica l for computer and human vision [18, 13, 20, 16, 7, 23].”
The Intrinsic Dimension of Images and Its Impact on Learning (2021, unknown): “It is widely believed that the combination of natural scenes and sensor properties yields very sparse and concentrated image distributions, as has been supported by several empirical studies on image patches (Lee et al., 2003; Donoho & Grimes, 2005; Carlsson et al., 2008).”
Random Walks on Simplicial Complexes and the normalized Hodge Laplacian (2018, background): “This is standard practice in persistent homology [29], where the data might be a time series of synaptic firings in a brain [57] or a set of images [84].”
Natural image denoising: Optimality and inherent bounds (2011, background): “Several works attempted to estimate the entropy of natural images [15, 4].”
Visual perception and the statistical properties of natural scenes. (2008, methodology): “Two popular approaches have been to apply either PCA or independent components analysis (ICA) to a collection of patches sampled from natural images (for another approach, see Lee et al. 2003).”
Barcodes: The persistent topology of data (2007, background): “We refer the interested reader to [18] for details.”
Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps With Accurate Object Boundaries (2018, background): “It is motivated by the previous studies on statistical properties of range images of natural scenes [15, 1, 19].”
Topology and data (2009, background): “The goal of the paper [41] is now to obtain some understanding of how this set sits within S(7), and what can be said about the patches which do appear.”
Mapping Natural Image Patches by Explicit and Implicit Manifolds (2007, background): “For example, a step edge is a low dimensional manifold in the image space [7].”
Vector Diffusion Maps and the Connection Laplacian (2011, unknown): “…maps and other non-linear dimensionality reduction methods are either directly or indirectly related to the heat kernel for functions over the data, ∗Department of Mathematics and PACM, Princeton University, Fine Hall, Washington Road, Princeton NJ 08544-1000 USA, email:…”
Local adaptivity to variable smoothness for exemplar-based image denoising and representation (2005, background): “image modeling [82, 46], image denoising [63], image reconstruction and super-resolution [29] and image rendering [28]).”
On Advances in Statistical Modeling of Natural Images (2004, methodology): “The image based search [51, 64] is also inspired by Marr’s idea [56] of representing early vision by converting an array of raw intensities into a symbolic representation—a so called “primal sketch”—with primitives such as edges, bars, blobs and terminations as basic elements.”