Supervised Dictionary Learning

By Julien Mairal, Francis R. Bach, Jean Ponce, Guillermo Sapiro, Andrew Zisserman. Neural Information Processing Systems, 2008

1,202 citations recorded by the source corpus.

It is now well established that sparse signal models are well suited for restoration tasks and can be effectively learned from audio, image, and video data. Recent research has been aimed at learning discriminative sparse models instead of purely reconstructive ones. This paper proposes a new step in that direction, with a novel sparse representation for signals belonging to different classes in terms of a shared dictionary and discriminative class models. The linear version of the proposed model admits a simple probabilistic interpretation, while its most general variant admits an interpretation in terms of kernels. An optimization framework for learning all the components of the proposed model is presented, along with experimental results on standard handwritten digit and texture classification tasks.

Subfields: Artificial Intelligence, Language and Linguistics

View Paper: https://www.semanticscholar.org/paper/9d65ba8bb20ae6dd001b9833c525c279dfe18916

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Language and Linguistics

Citations in the corpus

Information Constraints on Auto-Encoding Variational Bayes (2018, background): “For example, one may learn a low-dimensional representation that is predictive of a particular label y, as in supervised dictionary learning [8].”
Fisher Discrimination Dictionary Learning for sparse representation (2011, methodology): “Recently sparse representation techniques have also led to promising results in image classification, e.g. face recognition (FR) [5] [6] [7] [10] [31], digit and texture classification [8] [9] [11] [12], etc.”
Task-Driven Dictionary Learning (2010, methodology): “Our approach follows the formulation presented in [9], but is slightly different and falls into our task-driven dictionary learning framework.”
Subspace Interpolation via Dictionary Learning for Unsupervised Domain Adaptation (2013, methodology): “In this work, we use a dictionary to represent one domain, as dictionary learning based methods [1, 24] have recently become very popular for subspace modeling.”
Projective dictionary pair learning for pattern classification (2014, methodology): “Most of the state-of-the-art discriminative DL methods [5, 7, 9] can be formulated under the following framework: minD,A ‖ X − DA ‖(2)F +λ ‖ A ‖p +Ψ(D,A,Y), (1) where λ ≥ 0 is a scalar constant, Y represents the class label matrix of samples in X, D is the synthesis dictionary to be learned, and A i”
The Utility of Sparse Representations for Control in Reinforcement Learning (2018, methodology): “…some approaches developed to learning sparse representations incrementally, particularly through factorization approaches for dictionary learning (Mairal et al. 2009; Mairal et al. 2010; Le, Kumaraswamy, and White 2017) or for general sparse distributions (Ol-shausen and Field 1997; Olshausen…”
Online Learning for Matrix Factorization and Sparse Coding (2009, neutral): “For instance, in Mairal et al. (2009c), state-of-the-art results in image denoising and demosaicking are achieved with this formulation.”
Discriminative K-SVD for dictionary learning in face recognition (2010, background): “Other efforts along similar direction include [14] and [15], which use more sophisticated objective functions in dictionary optimization in the training stage in order to gain some discriminative power for the dictionary.”
Transfer Sparse Coding for Robust Image Representation (2013, background): “interest in machine learning, pattern recognition, signal processing [9, 11, 8], and has been successfully applied to image classification [22, 12, 24] and face recognition [21, 6].”
Semi-supervised Robust Dictionary Learning via Efficient l-Norms Minimization (2013, methodology): “Recently, the advances in sparse coding have shown that linearly decomposing a signal using a few atoms of a learned dictionary instead of a predefined one usually leads to state-ofthe-art performance for a number of practical computer vision applications [7, 13, 10, 15].”
Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis (2012, methodology): “Learning a dictionary from example image patches has been attracting much interest, and some representative methods have been proposed, such as K-SVD [1], supervised dictionary learning [19], online dictionary learning [17], etc.”
Sparse Modeling for Image and Vision Processing (2014, methodology): “To deal with unobserved information, the dictionary learning formulation can be modified by introducing a binary mask Mi for every patch indexed by i [Mairal et al., 2008c,e].”
Laplacian Sparse Coding, Hypergraph Laplacian Sparse Coding, and Applications (2013, background): “Apart from these, sparse coding has also been successfully applied in many computer vision applications, such as image classification [13], [14], face recognition [10], [9], image restoration [5], image annotation [11], [12], etc.”
Energy Disaggregation via Discriminative Sparse Coding (2010, methodology): “The algorithm is similar in spirit to a number of recent approaches to discriminative training of sparse representations [12, 17, 18].”
A Two-Phase Test Sample Sparse Representation Method for Use With Face Recognition (2011, methodology): “In [22] [23] [24] [25] [26] [27] [28], the extent of sparseness is evaluated by the 1 l norm of the coefficient vector of the linear combination and it is considered that a smaller 1 l norm means a higher extent of " sparseness " .”
Sparse Representation for Computer Vision and Pattern Recognition (2010, methodology): “…variations and extensions of 1 minimization have been applied to many vision tasks, including face recognition [71], image super-resolution [75], motion and data segmentation [33], [56], supervised denoising and inpainting [51] and background modeling [16], [21] and image classification [47], [48].”
From BoW to CNN: Two Decades of Texture Representation for Texture Classification (2018, methodology): “…the class speciﬁc codebooks were optimized for reconstruction, but signiﬁcant improvements have been shown by optimizing for discriminative power instead [45, 139, 140], an approach which is, however, associated with high computational cost, especially when the number of texture classes C is large.”