Learning Overcomplete Representations

By Michael S. Lewicki, Terrence J. Sejnowski. Neural Computation, 2000

1,291 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Signal Processing

View Paper: https://www.semanticscholar.org/paper/42d906c733f273109c0ed716a5ef6e2a379beb26

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Signal Processing

Citations in the corpus

Artifact Suppressed Dictionary Learning for Low-Dose CT Image Processing (2014, methodology): “The coefficients of the linear combination can be estimated through the sparse coding process described in [25].”
Blind source separation of more sources than mixtures using overcomplete representations (1999, methodology): “Lewicki and Sejnowski [11] approximated (5) by fitting a multivariate Gaussian around .”
Fourth-Order Cumulant-Based Blind Identification of Underdetermined Mixtures (2007, methodology): “Uniqueness of the source distributions allows for the joint estimation of sources and mixing matrix in a probabilistic framework [34].”
Bilinear Sparse Coding for Invariant Vision (2005, methodology): “In contrast, recent research on image coding has stressed the importance of localized, independent features derived from metrics that emphasize the higher-order statistics of inputs (Olshausen & Field, 1996, 1997; Bell & Sejnowski, 1997; Lewicki & Sejnowski, 2000).”
Sparse Feature Learning for Deep Belief Networks (2007, methodology): “Similarly to [15] we have collected test image codes after rectification (except for PCA which is linear), and we have measured the root mean square error (RMSE) and the entropy.”
Simple, Efficient, and Neural Algorithms for Sparse Coding (2015, background): “A related paper of the same authors Olshausen and Field (1997b) (and also Lewicki and Sejnowski (2000)) places sparse coding in a more familiar generative model setting whereby the data points y ( i ) ’s are assumed to be probabilistically generated according to a model y ( i ) = A ∗ · x ∗ ( i ) +…”
Exact Recovery of Sparsely Used Overcomplete Dictionaries (2013, background): “However, for most practical settings, it has been argued that overcomplete representations, where r ≫ d , are far more relevant, and can provide greater ﬂexibility in modeling as well as greater robustness to noise [25, 11].”
$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation (2006, methodology): “Both the BP and the FOCUSS can be motivated based on maximum a posteriori (MAP) estimation, and indeed several works used this reasoning directly [22]–[25].”
Random projection-based multiplicative data perturbation for privacy preserving distributed data mining (2006, background): “, the source signals are assumed to have sparse distribution [37].”
Non-local sparse models for image restoration (2009, background): “In turn, this requires demosaicking the raw signal—that is, reconstructing a full color image from the sensor’s RGB (Bayer) pattern—a difficult problem in itself.”
New Algorithms for Learning Incoherent and Overcomplete Dictionaries (2013, methodology): “(We will later discuss why current algorithms in [39], [22], [4], [36], [38] do not come with such guarantees.)”
A survey of dimensionality reduction techniques (2014, background): “Another possibility to learn the dictionary is by Maximum Likelihood [Lewicki2000].”
Learning Deep Architectures for AI (2007, methodology): “1) is convex, minimizing jointly the codes and the decoder bases W is not convex, but has been done successfully with many different algorithms [46, 58, 116, 121, 139, 148].”