Dictionaries for Sparse Representation Modeling

By Ron Rubinstein, Alfred Marcel Bruckstein, Michael Elad. Proceedings of the IEEE, 2010

1,356 citations recorded by the source corpus.

Sparse and redundant representation modeling of data assumes an ability to describe signals as linear combinations of a few atoms from a pre-specified dictionary. As such, the choice of the dictionary that sparsifies the signals is crucial for the success of this model. In general, the choice of a proper dictionary can be done using one of two ways: i) building a sparsifying dictionary based on a mathematical model of the data, or ii) learning a dictionary to perform best on a training set. In this paper we describe the evolution of these two paradigms. As manifestations of the first approach, we cover topics such as wavelets, wavelet packets, contourlets, and curvelets, all aiming to exploit 1-D and 2-D mathematical models for constructing effective dictionaries for signals and images. Dictionary learning takes a different route, attaching the dictionary to a set of examples it is supposed to serve. From the seminal work of Field and Olshausen, through the MOD, the K-SVD, the Generalized PCA and others, this paper surveys the various options such training has to offer, up to the most recent contributions and structures.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/18a1a63282e1c6bc2851ff435551444a11a5b81d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Analysis K-SVD: A Dictionary-Learning Algorithm for the Analysis Sparse Model (2013, neutral): “While this description may seem similar to the synthesis counterpart approach, it is in-fact very different when dealing with a redundant dictionary p > d .”
Recent Advances in Open Set Recognition: A Survey (2018, background): “Sparse Representation-based: In recent years, the sparse representation-based techniques have been widely used in computer vision and image processing fields [99], [100], [101].”
Metaface learning for sparse representation based face recognition (2010, methodology): “Output that identity(y)=argmin ri(y).”
High-resolution hyperspectral imaging via matrix factorization (2011, background): “Yhs into A and Q as an instance of the “dictionary learning” problem discussed in the survey paper [ 26 ] and references therein.”
Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization (2010, methodology): “Recently, there has been much effort in learning dictionaries from example image patches [13]–[15], [26]–[31], [55], leading to state-of-the-art”
Learning Graphs From Data: A Signal Representation Perspective (2018, background): “Dictionary learning [51], [52] is an area of research in signal processing and machine learning where the signals are represented as a linear combination of simple components, i.”
Joint Bi-layer Optimization for Single-Image Rain Streak Removal (2017, methodology): “Dictionary-based sparse prior [23] describes an image patch as a linear combination of a few atoms from a pre-specified dictionary.”
On the Identifiability of Overcomplete Dictionaries via the Minimisation Principle Underlying K-SVD (2013, background): “…the research focus in dictionary learning has been on algorithmic development, meaning that by now there are several dictionary learning algorithms, which are efﬁcient in practice and therefore popular in applications, see [13], [19], [1], [22], [34], [20], [29] or [26] for a more complete survey.”
Vertex-Frequency Analysis on Graphs (2013, background): “…fundamental problems of signal processing, and the literature is ﬁlled with a wide range of dictionaries, including, e.g., Fourier, time-frequency, curvelet, shearlet, and bandlet dictionaries (see, e.g., [1] for an excellent historical overview of dictionary design methods and signal transforms).”
Sparse Representation-Based Open Set Recognition (2017, background): “IN recent years, sparse representation-based techniques have drawn much interest in computer vision and image processing fields [1], [2].”
The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains (2012, background): “…to sparsely represent different classes of high-dimensional data such as audio signals and images that lie on regular Euclidean spaces has led to a number of resounding successes in the aforementioned signal processing tasks (see, e.g., [13, Section II] for a recent survey of transform methods).”
Bayesian sparse representation for hyperspectral image super resolution (2015, methodology): “We adopt these conventions from the sparse representation literature [24].”
Sparse Representation Based Image Interpolation With Nonlocal Autoregressive Modeling (2013, background): “12 dictionaries learned from example images can lead to better image restoration results [23, 30, 39].”
Color demosaicking by local directional interpolation and nonlocal adaptive thresholding (2011, methodology): “The dictionary learning [37-38] methods have been recently proposed to learn an over-complete dictionary of bases from a training dataset to span the sparse domain.”