Shiftable multiscale transforms

By Eero P. Simoncelli, William T. Freeman, Edward H. Adelson, David J. Heeger. 1992

One of the major drawbacks of orthogonal wavelet transforms is their lack of translation invariance: the content of wavelet subbands is unstable under translations of the input signal. Wavelet transforms are also unstable with respect to dilations of the input signal and, in two dimensions, rotations of the input signal. The authors formalize these problems by defining a type of translation invariance called shiftability. In the spatial domain, shiftability corresponds to a lack of aliasing; thus, the conditions under which the property holds are specified by the sampling theorem. Shiftability may also be applied in the context of other domains, particularly orientation and scale. Jointly shiftable transforms that are simultaneously shiftable in more than one domain are explored. Two examples of jointly shiftable transforms are designed and implemented: a 1-D transform that is jointly shiftable in position and scale, and a 2-D transform that is jointly shiftable in position and orientation. The usefulness of these image representations for scale-space analysis, stereo disparity measurement, and image enhancement is demonstrated. >

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Making Convolutional Networks Shift-Invariant Again (2019, background): “As discussed in Simoncelli et al. (1992), literal shift-equivariance cannot hold when subsampling.”
Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks (2015, methodology): “ture synthesis [3, 33, 22], with Portilla & Simoncelli [22] making use of a steerable pyramid wavelet representation [27], similar to our use of a Laplacian pyramid.”
Sparse coding with an overcomplete basis set: a strategy employed by V1? (1997, background): “This point has been emphasized by Simoncelli, Freeman, Adelson, & Heeger (1992) , who show that overcomplete codes allow for small translations or scaling of local image features to result in a smooth and graceful change in the distribution of activity among coefficients.”
Why do deep convolutional networks generalize so poorly to small image transformations? (2018, background): “[38] who wrote "We cannot literally expect translation invariance in a system based on convolution and subsampling: translation of the input signal cannot produce simple translations of the transform coefficients, unless the translation is a multiple of each of the subsampling factors in the system"”
Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling (1998, methodology): “The filtering theory developed along this direction includes the Gabor filters (Gabor, 1946; Daugman, 1985) and wavelet pyramids (Mallat, 1989; Simoncelli et al., 1992; Coifman and Wickerhauser, 1992; Donoho and Johnstone, 1994).”
Polar Transformer Networks (2017, methodology): “An exact steerability framework was started in [6] where rotational steerability for Gaussian derivatives was explicitly computed, and was extended to the shiftable pyramid [24] handling rotations and scale.”
CubeNet: Equivariance to 3D Rotation and Translation (2018, background): “[44], and have been placed on firm theoretical ground by Teo [47] in his PhD thesis.”
Learning a No-Reference Quality Metric for Single-Image Super-Resolution (2016, methodology): “We first apply the steerable pyramid decomposition [37] on an SR image to generate neighboring wavelet coefficients.”
Geodesic Active Regions and Level Set Methods for Supervised Texture Segmentation (2002, background): “A quite common filter bank selection is Gabor filters (Gabor, 1946; Bovik et al., 1990; Dunn and Higgins, 1995), or the wavelet transform (Mallat, 1989; Simoncelli et al., 1992) which is usually applied in a pyramidstructure form (Chang and Kuo, 1993; Laine and Fan, 1993; Unser, 1995).”