T-SNE-CUDA: GPU-Accelerated T-SNE and its Applications to Modern Data

By David Chan, Roshan Rao, Forrest Huang, John F. Canny. 2018

106 citations recorded by the source corpus.

Modern datasets and models are notoriously difficult to explore and analyze due to their inherent high dimensionality and massive numbers of samples. Existing visualization methods which employ dimensionality reduction to two or three dimensions are often inefficient and/or ineffective for these datasets. This paper introduces T-SNE-CUDA, a GPU-accelerated implementation of t-distributed Symmetric Neighbour Embedding (t-SNE) for visualizing datasets and models. T-SNE-CUDA significantly outperforms current implementations with 50-700x speedups on the CIFAR-10 and MNIST datasets. These speedups enable, for the first time, visualization of the neural network activations on the entire ImageNet dataset - a feat that was previously computationally intractable. We also demonstrate visualization performance in the NLP domain by visualizing the GloVe embedding vectors. From these visualizations, we can draw interesting conclusions about using the L2 metric in these embedding spaces. T-SNE-CUDA is publicly available at https://github.com/CannyLab/tsne-cuda.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xiang Ren
2. J. Snyder-Cappione
3. Cong Fu
4. David Chan
5. Philipp Berens
6. Deng Cai
7. John F. Canny
8. Yonghui Zhang

In Computer Vision and Pattern Recognition

1. Deng Cai
2. Philipp Berens
3. Dmitry Kobak
4. Xiang Ren
5. Dacheng Tao
6. David Chan
7. Roshan Rao
8. Forrest Huang

Citations in the corpus

• The art of using t-SNE for single-cell transcriptomics (2018, background): “There is some recent work on how to accelerate t-SNE computations by using GPUs (Chan et al., 2018; Pezzotti et al., 2018), but we did not explore these implementations here.”
• AtSNE: Efficient and Robust Visualization on GPU through Hierarchical Optimization (2019, methodology): “We use the default parameters suggested by BH-t-SNE [19], t-SNE-CUDA[2], and LargeVis [15].”
• GPU accelerated t-distributed stochastic neighbor embedding (2019, methodology): “[36], show that human-in-the-loop training using interactive realtime visualization and targeting techniques can be an interesting and promising area of research.”
• Automated Optimal Parameters for T-Distributed Stochastic Neighbor Embedding Improve Visualization and Allow Analysis of Large Datasets (2018, methodology): “Therefore, we expect opt-SNE to be applicable for existing or future adaptations of t-SNE even if alternative methods of computation are utilized 20,36 provided that they retain the core principles of t-SNE embedding.”