Nonlinear dimensionality reduction by locally linear embedding.

By Sam T. Roweis, Lawrence K. Saul. 2000

16,021 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

A Generative Process for Contractive Auto-Encoders (2012, background): “…Analysis (PCA) discovers a linear manifold near which the density may concentrate, Non-Linear Manifold Learning algorithms (Sch¨olkopf et al., 1998; Roweis and Saul, 2000; Tenenbaum et al., 2000) attempt to discover the structure of manifolds (which may be non-linear) near which the true density…”
Graph Attention Networks (2017, background): “Other related approaches include locally linear embedding (LLE) (Roweis & Saul, 2000) and memory networks (Weston et al., 2014).”
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles (2016, background): “If the data structure suggests that data points might concentrate around a manifold, then manifold learning techniques can be employed [29,4].”
Generative Modeling by Estimating Gradients of the Data Distribution (2019, background): “This hypothesis empirically holds for many datasets, and has become the foundation of manifold learning [3, 47].”
node2vec: Scalable Feature Learning for Networks (2016, methodology): “…linear and non-linear dimensionality reduction techniques such as Principal Component Analysis, Multi-Dimensional Scaling and their extensions [3, 27, 30, 35] optimize an objective that transforms a representative data matrix of the network such that it maximizes the variance of the data…”
LINE: Large-scale Information Network Embedding (2015, methodology): “Because of this importance, many existing graph embedding algorithms such as IsoMap, LLE, Laplacian eigenmap, and graph factorization have the objective to preserve the ﬁrst-order proximity.”
Adversarial Autoencoders (2015, methodology): “In Section 3, we will show that the test-likelihood achieved by the joint training scheme of adversarial autoencoders outperforms the test-likelihood of GMMN and GMMN+AE on MNIST and Toronto Face datasets.”
Representation Learning: A Review and New Perspectives (2012, methodology): “…built on this geometric perspective adopt a non-parametric approach, based on a training set nearest neighbor graph (Schölkopf et al., 1998; Roweis and Saul, 2000; Tenenbaum et al., 2000; Brand, 2003; Belkin and Niyogi, 2003; Donoho and Grimes, 2003; Weinberger and Saul, 2004; Hinton and…”
A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines (2018, background): “In addition, Locally Linear Embedding (LLE) [104] has a similar goal, to learn a manifold which preserves neighbors, but a very diﬀerent approach: it linearly reconstructs each point from its neighbors in order to maintain the local structure of the manifold.”
Relational autoencoder for feature extraction (2017, background): “Locally Linear Embedding (LLE) [19] preserves data relationships by embedding local neighbourhood when mapping to low-dimensional space.”
Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization (2020, methodology): “More traditional methods that focus on local structure include LLE (Roweis and Saul, 2000), Isomap (Tenenbaum et al.”
Nonparametric Divergence Estimation with Applications to Machine Learning on Distributions (2011, methodology): “Low-dimensional embedding and manifold learning (Roweis and Saul, 2000) are well studied problems; several diﬀerent algorithms have been proposed for this problem (Borg and Groenen, 2005, Tenenbaum et al., 2000, Sun et al., 2010, Zhang and Zha, 2004, Belkin and Niyogi, 2003, Donoho and Grimes,…”
FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling (2018, background): “A major category consists of factorization based algorithms that yield the embedding through matrix factorizations; see, e.g., Roweis & Saul (2000); Belkin & Niyogi (2001); Ahmed et al. (2013); Cao et al. (2015); Ou et al. (2016).”
Geometric Deep Learning: Going beyond Euclidean data (2016, background): “Examples of manifold learning include different flavors of multidimensional scaling (MDS) [26], locally linear embedding (LLE) [27], stochastic neighbor embedding (t-SNE) [28], spectral embeddings such as Laplacian eigenmaps [29] and diffusion maps [30], and deep models [31].”
A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications (2017, background): “(5) + O(community detection) + (community proximity constraint) ULGE [2] W ¼ ZD 1ZT , where Z ¼ argminzT i 1¼1;zi 0 Pm j¼1 kXi ujk 2 2zij þ g Pm j¼1 z 2 ij a ¼ argminkaTX Fpk2F þ akak 2 F whereFp is the top p eigenvectors ofW LLE [102] KNN,W ¼ argmin P i kXi P j WijXjk 2 y ¼ argmin P i kyi P j Wijyj”
Deep Learning on Lie Groups for Skeleton-Based Action Recognition (2016, methodology): “In particular, inspired by the classical manifold learning theory [38, 36, 4, 12, 20, 19], we equip the new network structure with rotation mapping layers, with which the input Lie group features are transformed to new ones with better alignment.”