From Information Scaling of Natural Images to Regimes of Statistical Models

By Ying Nian Wu, Song-Chun Zhu, Cheng-en Guo. 2007

49 citations recorded by the source corpus.

Computer vision can be considered a highly specialized data collection and data analysis problem. We need to understand the special properties of image data in order to construct statistical models for representing the wide variety of image patterns. One special property of vision that distinguishes itself from other sensory data such as speech data is that distance or scale plays a profound role in image data. More specifically, visual objects and patterns can appear at a wide range of distances or scales, and the same visual pattern appearing at different distances or scales produces different image data with different statistical properties, thus entails different regimes of statistical models. In particular, we show that the entropy rate of the image data changes over the viewing distance (as well as the camera resolution). Moreover, the inferential uncertainty changes with viewing distance too. We call these changes information scaling. From this perspective, we examine both empirically and theoretically two prominent and yet largely isolated research themes in image modeling literature, namely, wavelet sparse coding and Markov random fields. Our results indicate that the two models are appropriate on two different entropy regimes: sparse coding targets the low entropy regime, whereas the random fields are suitable for the high entropy regime. Because of information scaling, both models are necessary for representing and interpreting image intensity patterns in the whole entropy range, and information scaling triggers transitions between these two regimes of models. This motivates us to propose a full-zoom primal sketch model that integrates both sparse coding and Markov random fields. In this model, local image intensity patterns are classified into “sketchable regime” and “non-sketchable regime” by a sketchability criterion. In the sketchable regime, the image data are represented deterministically by highly parametrized sketch primitives. In the non-sketchable regime, the image data are characterized by Markov random fields whose sufficient statistics summarize computational results from failed attempts of sparse coding. The contribution of our work is two folded. First, information scaling provides a dimension to chart the space of natural images. Second, the full-zoom modeling scheme provides a natural integration of sparse coding and Markov random fields, thus enables us to develop a new and richer class of statistical models.

Subfields: Computer Vision and Pattern Recognition, Scaling & Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Song-Chun Zhu
2. Ying Nian Wu
3. Erik Nijkamp
4. Jianwen Xie
5. Mitch Hill
6. Yang Lu
7. Zhangzhang Si
8. Yilun Du

In Scaling & Learning Theory

1. Zhangzhang Si
2. Song-Chun Zhu
3. Ying Nian Wu
4. Wenze Hu
5. Yining Hong
6. Jianwen Xie

Citations in the corpus

• A Theory of Generative ConvNet (2016, background): “Another justification for the Gaussian white noise distribution is that it is the limiting distribution if we zoom out a texture image due to the central limit theorem, a phenomenon called information scaling by [34].”
• Mixtures of Conditional Gaussian Scale Mixtures Applied to Multiscale Image Representations (2011, neutral): “, who showed that many natural images are more difficult to compress at larger scales and argued that the entropy rate of natural images has to increase with scale [19].”
• Building a Telescope to Look Into High-Dimensional Image Spaces (2018, background): “As image scale increases, the number of recognizable image groups tends to increase until one reaches the threshold of perceptibility, where texton-scale images transition into texture-scale images and humans begin to lose the ability to identify distinguishing features [47].”
• Learning Active Basis Model for Object Detection and Recognition (2010, background): “This work is a continuation of our search for generative models of visual patterns, as well as our attempt to understand these models within a common information-theoretical framework (Wu et al. 2008).”
• Unsupervised learning of compositional sparse code for natural image representation (2013, methodology): “The exponential family model can be justified by the maximum entropy principle [16, 28, 36, 43].”