Stochastic Neighbor Embedding under f-divergences
By Daniel Jiwoong Im, Nakul Verma, Kristin Branson. 2018
23 citations recorded by the source corpus.
The t-distributed Stochastic Neighbor Embedding (t-SNE) is a powerful and popular method for visualizing high-dimensional data. It minimizes the Kullback-Leibler (KL) divergence between the original and embedded data distributions. In this work, we propose extending this method to other f-divergences. We analytically and empirically evaluate the types of latent structure-manifold, cluster, and hierarchical-that are well-captured using both the original KL-divergence as well as the proposed f-divergence generalization, and find that different divergences perform better for different types of structure. A common concern with $t$-SNE criterion is that it is optimized using gradient descent, and can become stuck in poor local minima. We propose optimizing the f-divergence based loss criteria by minimizing a variational bound. This typically performs better than optimizing the primal form, and our experiments show that it can improve upon the embedding results obtained from the original $t$-SNE criterion as well.
Subfields: Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- J. Snyder-Cappione
- Philipp Berens
- J. Böhm
- George C. Linderman
- Stefan Steinerberger
- Josef Spidlen
- Richard L. Halpert
- Yuval Kluger
Citations in the corpus
- Automated optimized parameters for T-distributed stochastic neighbor embedding improve visualization and analysis of large datasets (2019, background): “It is advisable to note that certain data structures, such as cluster hierarchy, cannot be revealed with t-SNE 43,44 . t-SNE accessibility in cytometry analysis software lead to its not infrequent misuse with cytometry data, evident when the cluster-like structure is not prominent in the map.”
- A Unifying Perspective on Neighbor Embeddings along the Attraction-Repulsion Spectrum (2020, background): “Related work Various trade-offs in SNE and t-SNE generalizations have been studied previously (Yang et al., 2009; Kobak et al., 2020; Venna et al., 2010; Amid et al., 2015; Amid and Warmuth, 2019; Narayan et al., 2015; Im et al., 2018), but our work is the first to study the exaggeration-induced tra”
- Heavy-tailed kernels reveal a finer cluster structure in t-SNE visualisations (2019, background): “5 (Figure 7), but it may not be the ideal metric to quantify the embedding quality [6].”
- Automated Optimal Parameters for T-Distributed Stochastic Neighbor Embedding Improve Visualization and Allow Analysis of Large Datasets (2018, background): “It is advisable to note that certain data structures, such as cluster hierarchy, cannot be revealed with t-SNE 40,41 . t-SNE accessibility in cytometry analysis software lead to its not infrequent misuse with cytometry data, evident when the cluster-like structure is not prominent in the map.”