Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization

By Yingfan Wang, Ying-Fan Wang, Haiyang Huang, Cynthia Rudin, Yaron Shaposhnik. 2020

528 citations recorded by the source corpus.

Dimension reduction (DR) techniques such as t-SNE, UMAP, and TriMAP have demonstrated impressive visualization performance on many real world datasets. One tension that has always faced these methods is the trade-off between preservation of global structure and preservation of local structure: these methods can either handle one or the other, but not both. In this work, our main goal is to understand what aspects of DR methods are important for preserving both local and global structure: it is difficult to design a better method without a true understanding of the choices we make in our algorithms and their empirical impact on the lower-dimensional embeddings they produce. Towards the goal of local structure preservation, we provide several useful design principles for DR loss functions based on our new understanding of the mechanisms behind successful DR methods. Towards the goal of global structure preservation, our analysis illuminates that the choice of which components to preserve is important. We leverage these insights to design a new algorithm for DR, called Pairwise Controlled Manifold Approximation Projection (PaCMAP), which preserves both local and global structure. Our work provides several unexpected insights into what design choices both to make and avoid when constructing DR algorithms.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lesia Semenova
2. Chudi Zhong
3. Zhi Chen
4. Haiyang Huang
5. Cynthia Rudin
6. Tong Wang
7. Octavio Loyola-González
8. Berk Ustun

In Computer Vision and Pattern Recognition

1. Zhi Chen
2. Lesia Semenova
3. Chudi Zhong
4. Haiyang Huang
5. Cynthia Rudin

In Representation & Self-Supervised Learning

1. Lesia Semenova
2. Chudi Zhong
3. Zhi Chen
4. Haiyang Huang
5. Cynthia Rudin

In Evaluation & Benchmarks

1. Lesia Semenova
2. Zhi Chen
3. Chudi Zhong
4. Haiyang Huang
5. Cynthia Rudin

Citations in the corpus

• Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges (2021, methodology): “Wang et al. (2020a) discussed the problem of numerical stiffness in gradient backpropagation during training of a PINN.”
• NOMAD Projection (2025, unknown): “We initialize our projection with PCA, as it has been found to improve global structure [27].”