GPU accelerated t-distributed stochastic neighbor embedding

By David Chan, Roshan Rao, Forrest Huang, John F. Canny. 2019

79 citations recorded by the source corpus.

Abstract Modern datasets and models are notoriously difficult to explore and analyze due to their inherent high dimensionality and massive numbers of samples. Existing visualization methods which employ dimensionality reduction to two or three dimensions are often inefficient and/or ineffective for these datasets. This paper introduces t-SNE-CUDA, a GPU-accelerated implementation of t-Distributed Symmetric Neighbor Embedding (t-SNE) for visualizing datasets and models. t-SNE-CUDA significantly outperforms current implementations with 15-700x speedups on the CIFAR-10 and MNIST datasets. These speedups enable, for the first time, large scale visualizations of modern computer vision datasets such as ImageNet, as well as larger NLP datasets such as GloVe. From these new visualizations, we can draw a number of interesting conclusions. In addition, the performance on machine learning datasets allows us to compute t-SNE embeddings in close to real time, and we explore the applications of such fast embeddings in the domain of importance sampling for neural network training.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In ML Systems & Hardware

Citations in the corpus

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, neutral): “This assumption may not hold true for some data, for instance, the weights and the activation in a neural network (see Chan et al., 2019, for a detailed example of such failure).”
Automated optimized parameters for T-distributed stochastic neighbor embedding improve visualization and analysis of large datasets (2019, methodology): “Therefore, we expect opt-SNE to be applicable for existing or future adaptations of t-SNE even if alternative methods of computation are utilized 37,38 provided that they retain the core principles of t-SNE embedding.”
The art of using t-SNE for single-cell transcriptomics (2018, methodology): “That said, the exact runtime will depend on the details of implementation, and both methods may be further accelerated in future releases, or by using GPU parallelisation 40 .”
Artificial intelligence in marketing: Topic modeling, scientometric analysis, and research agenda (2020, methodology): “Connectedness, Correspondence, and Relative Significance of the Topics Due to its high dimensionality (i.e., number of coordinates needed to locate objects in multidimensional space) and large samples, natural language is difficult to analyze objectively (Belkina et al., 2018; Chan et al., 2019).”
FedBABU: Towards Enhanced Representation for Federated Image Classification (2021, unknown): “For speed acceleration, we used t-SNE-CUDA (Chan et al., 2019).”