Prior Learning and Gibbs Reaction-Diffusion

By Song-Chun Zhu, David Mumford. 1997

This article addresses two important themes in early visual computation: it presents a novel theory for learning the universal statistics of natural images, and, it proposes a general framework of designing reaction-diffusion equations for image processing. We studied the statistics of natural images including the scale invariant properties, then generic prior models were learned to duplicate the observed statistics, based on minimax entropy theory. The resulting Gibbs distributions have potentials of the form U(I; /spl Lambda/, S)=/spl Sigma//sub /spl alpha/=1//sup k//spl Sigma//sub x,y//spl lambda//sup (/spl alpha/)/((F/sup (/spl alpha/)/*I)(x,y)) with S={F/sup (1)/, F/sup (2)/,...,F/sup (K)/} being a set of filters and /spl Lambda/={/spl lambda//sup (1)/(),/spl lambda//sup (2)/(),...,/spl lambda//sup (K)/()} the potential functions. The learned Gibbs distributions confirm and improve the form of existing prior models such as line-process, but, in contrast to all previous models, inverted potentials were found to be necessary. We find that the partial differential equations given by gradient descent on U(I; /spl Lambda/, S) are essentially reaction-diffusion equations, where the usual energy terms produce anisotropic diffusion, while the inverted energy terms produce reaction associated with pattern formation, enhancing preferred image features. We illustrate how these models can be used for texture pattern rendering, denoising, image enhancement, and clutter removal by careful choice of both prior and data models of this type, incorporating the appropriate features.

Subfields: Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Probabilistic & Bayesian Methods

Citations in the corpus

GAN Inversion: A Survey (2021, methodology): “It has been widely adopted in computer vision tasks, including MRF [196], [197], [198], dark channel prior [199] and total variation regularizer [200], which all model correlation among neighboring pixels via Gibbs distribution.”
Learning FRAME Models Using CNN Filters (2015, methodology): “the Langevin dynamics starting from the images generated by the previous iteration of the learning algorithm, a scheme called “warm start” in the literature. The Langevin equation was also adopted by Zhu and Mumford (1997), who called the corresponding gradient descent algorithm the Gibbs reaction a”
On deep generative models with applications to recognition (2011, background): “The most intuitive way to test a generative model is to draw samples from it [25].”
Fields of Experts: a framework for learning image priors (2005, background): “As observed by Zhu and Mumford [26], this is related to non-linear diffusion methods.”
Sparse Representation for Color Image Restoration (2008, methodology): “Such is the case with the MRF prior learned in [27] and later in [28] and [29]; the wavelet based image prior as appears in [30]; the Tikhonov regularization proposed by Haber and Tenorio [31]; the super-resolution approach adopted by Baker and Kanade [32]; and the recent K-SVD denoising as…”
Multi-Stage Progressive Image Restoration (2021, unknown): “In order to restrict the solution space to valid/natural images, existing restoration techniques [19, 29, 39, 59, 66, 67, 100] explicitly use image priors that are handcrafted with empirical observations.”
Blind deconvolution using a normalized sparsity measure (2011, background): “Zhu and Mumford [26] learn arbitrary energy functions for a set of oriented derivative filters via Gibbs sampling in a maximum likelihood approach.”
From Information Scaling of Natural Images to Regimes of Statistical Models (2007, background): “Zhu and Mumford (1997) connected Markov random fields to partial differential equations and variational approaches for image processing (Aubert and Kornprobst, 2002).”
Occlusion Models for Natural Images: A Statistical Study of a Scale-Invariant Dead Leaves Model (2004, methodology): “In this section, we use a different formalism— the full probability measure of a Poisson process—to show that higher-order scaling, which has been observed empirically in the filter responses of natural images (Ruderman, 1994; Zhu and Mumford, 1997 ), places further restrictions on the images.”
Markov Random Field Modeling in Image Analysis (2001, methodology): “The FRAME (Filter, Random Fields And Maximum Entropy) model, proposed by (Zhu et al. 1997; Zhu and Mumford 1997; Zhu et al. 1998), is a generalized MRF model which fuses the essence of filtering theory and MRF modeling through the maximum entropy principle.”
Generating more realistic images using gated MRF's (2010, background): “Indeed, several authors [10, 7] have proposed that these models should be evaluated by generating images and”
Minimax Entropy Principle and Its Application to Texture Modeling (1997, background): “As an illustration of the nongaussian property, Figure 1a shows the empirical marginal distribution (or histogram) of the intensity differences of horizontally adjacent pixels of some natural images (Zhu & Mumford, 1997).”
Object perception as Bayesian inference. (2004, background): “Zhu & Mumford (1997) have derived a strong generative model they call a \generic image prior" based on (cid:12)lter histogram statistics.”
Internal statistics of a single natural image (2011, background): “, [16, 13, 15]) impose parametric distribution on natural image responses to local filters.”
Statistical Modeling and Conceptualization of Visual Patterns (2003, methodology): “Let h obs be the normalized histogram averaged over three scales and impose constraints that a model p should produce the same histograms (marginal distributions), E p hðr x I ðsÞ Þ h i ¼ E p hðr y I ðsÞ Þ h i ¼ h obs ; s ¼ 0; 1; 2; 3: ð10Þ Zhu and Mumford [94] derived a descriptive model, p inv ðI;”