Visualizing Large-scale and High-dimensional Data

By Jian Tang, Jian Tang, Jingzhou Liu, Ming Zhang, Qiaozhu Mei. 2016

426 citations recorded by the source corpus.

We study the problem of visualizing large-scale and high-dimensional data in a low-dimensional (typically 2D or 3D) space. Much success has been reported recently by techniques that first compute a similarity structure of the data points and then project them into a low-dimensional space with the structure preserved. These two steps suffer from considerable computational costs, preventing the state-of-the-art methods such as the t-SNE from scaling to large-scale and high-dimensional data (e.g., millions of data points and hundreds of dimensions). We propose the LargeVis, a technique that first constructs an accurately approximated K-nearest neighbor graph from the data and then layouts the graph in the low-dimensional space. Comparing to t-SNE, LargeVis significantly reduces the computational cost of the graph construction step and employs a principled probabilistic model for the visualization step, the objective of which can be effectively optimized through asynchronous stochastic gradient descent with a linear time complexity. The whole procedure thus easily scales to millions of high-dimensional data points. Experimental results on real-world data sets demonstrate that the LargeVis outperforms the state-of-the-art methods in both efficiency and effectiveness. The hyper-parameters of LargeVis are also much more stable over different data sets.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction (2018, methodology): “Following both [54] and [41], we take a sampling based approach to the optimization.”
Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “Prominent examples include LargeVis (Tang et al., 2016), which is widely used in network analysis, and UMAP (McInnes et al.”
Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization (2020, methodology): “Here, each algorithm exhibits its typical characteristics. t-SNE, and its more modern sister LargeVis (Tang et al., 2016), produces spurious clusters, making its results untrustworthy.”
TriMap: Large-scale Dimensionality Reduction Using Triplets (2019, methodology): “Unlike PCA, much of the focus of the more recent non-linear methods, including t-SNE (Maaten & Hinton, 2008), LargeVis (Tang et al., 2016), and UMAP (McInnes et al.”
Label Informed Attributed Network Embedding (2017, neutral): “The learned embedding representation paves the way for numerous applications such as node classiﬁcation [33, 47], link prediction [10, 31], and network visualization [35].”
Parametric UMAP Embeddings for Representation and Semisupervised Learning (2020, methodology): “…inspired embedding algorithms. e In contrast, UMAP is optimized using negative sampling (Mikolov, Sutskever, Chen, Corrado, & Dean, 2013; Tang et al., 2016) to sparsely sample edges during optimization, making it, in principle, better suited to batch-wise training as is common in deep…”
Linear tSNE optimization for the Web (2018, methodology): “Computationally, UMAP follows very closely LargeVis and adopts a kNN descent procedure [6] and Stochastic Gradient Descent minimization of the objective function.”
Data Poisoning Attack against Unsupervised Node Embedding Methods (2018, background): “, 2014), link prediction (Grover & Leskovec, 2016), and visualization (Tang et al., 2016).”
Heavy-tailed kernels reveal a finer cluster structure in t-SNE visualisations (2019, methodology): “T-distributed stochastic neighbour embedding (t-SNE) [12] and related methods [15,13] are used for data visualisation in many scientific fields dealing with thousands or even millions of high-dimensional samples.”
Accelerated Hierarchical Density Based Clustering (2017, background): “The effects of approximate nearest neighbor search via spill trees [49], bounding adjustments [14], or RP-trees with local neighborhood exploration [50], both on core-distance computation, and within March’s algorithm remains unexplored.”
Fully-automated and ultra-fast cell-type identification using specific marker combinations from single-cell transcriptomic data (2022, methodology): “The clusters are visualized using either principal components analysis (PCA), t-distributed Stochastic Neighbor Embedding (t-SNE), Uniform Manifold Approximation and Projection for Dimension Reduction (UMAP), Isomap 55 , Diffusion Map 56 , largeVis 57 or by means of expression heatmaps.”
Generalizable and Scalable Visualization of Single-Cell Data Using Neural Networks (2018, background): “Although a number of recent studies introduced new techniques for improving the scalability of data visualization tools (Dzwinel and Wcis1o, 2015; Tang et al., 2016), they do not address the lack of generalizability that net-SNE overcomes.”
Large-scale generation and analysis of filamentous fungal DNA barcodes boosts coverage for kingdom fungi and reveals thresholds for fungal species and higher taxon delimitation (2018, methodology): “2018) to compute a complete similarity matrix and LargeVis (Tang et al. 2016) to calculate the coordinates of the sequences.”
A more globally accurate dimensionality reduction method using triplets (2018, methodology): “LargeVis is a more recent method (Tang et al., 2016) that also aims to preserve the pairwise (dis)similarities of the points in the high-dimensional space in the low-dimensional embedding.”
Adversarial Network Embedding (2017, background): “Network visualization is an indispensable way to analyze high-dimensional graph data, which can help reveal intrinsic structure of the data intuitively, and thus enables human to gain insight into the data, draw some initial conclusions, and facilitates further quantitative analysis (Tang et al. 201”
A Unifying Perspective on Neighbor Embeddings along the Attraction-Repulsion Spectrum (2020, methodology): “Dropping constant terms, we obtain which is the same loss function as the one introduced earlier by LargeVis (Tang et al., 2016).”
Toward a Quantitative Survey of Dimension Reduction Techniques (2019, background): “As such, high-dimensional visualization has become an important sub-ﬁeld of Information Visualization (InfoVis) [1], [2], [3], [4].”