Heavy-tailed kernels reveal a finer cluster structure in t-SNE visualisations

By Dmitry Kobak, George C. Linderman, Stefan Steinerberger, Yuval Kluger, Philipp Berens. 2019

46 citations recorded by the source corpus.

T-distributed stochastic neighbour embedding (t-SNE) is a widely used data visualisation technique. It differs from its predecessor SNE by the low-dimensional similarity kernel: the Gaussian kernel was replaced by the heavy-tailed Cauchy kernel, solving the 'crowding problem' of SNE. Here, we develop an efficient implementation of t-SNE for a t-distribution kernel with an arbitrary degree of freedom ν, with ν → ∞ corresponding to SNE and ν = 1 corresponding to the standard t-SNE. Using theoretical analysis and toy examples, we show that ν < 1 can further reduce the crowding problem and reveal finer cluster structure that is invisible in standard t-SNE. We further demonstrate the striking effect of heavier-tailed kernels on large real-life data sets such as MNIST, single-cell RNA-sequencing data, and the HathiTrust library. We use domain knowledge to confirm that the revealed clusters are meaningful. Overall, we argue that modifying the tail heaviness of the t-SNE kernel can yield additional insight into the cluster structure of the data.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Philipp Berens
2. Blaz Zupan
3. J. Böhm
4. Pavlin Gregor Policar
5. Martin Stražar
6. Dmitry Kobak
7. R. Bradford
8. B. Bruce

In Computer Vision and Pattern Recognition

1. Philipp Berens
2. Blaz Zupan
3. J. Böhm
4. Pavlin Gregor Policar
5. Martin Stražar
6. Dmitry Kobak

In Molecular Biology

1. Dmitry Kobak
2. J. Böhm
3. Philipp Berens
4. Pavlin Gregor Policar
5. Martin Stražar
6. Blaz Zupan
7. George C. Linderman
8. R. Bradford

Citations in the corpus

• openTSNE: a modular Python library for t-SNE dimensionality reduction and embedding (2019, background): “Kobak and Berens (2019) suggest that increasing attractive forces between similar data points controlled via the exaggeration parameter leads to more compact clusters and, subsequently, more informative visualizations.”
• A Unifying Perspective on Neighbor Embeddings along the Attraction-Repulsion Spectrum (2020, background): “In particular, the tail-heaviness of the low-dimensional similarity kernel (Yang et al., 2009) controls the emphasis put on the fine cluster structure of the data (Kobak et al., 2020).”