The Platonic Representation Hypothesis

By Minyoung Huh, Brian Cheung, Tongzhou Wang, Phillip Isola. International Conference on Machine Learning, 2024

392 citations recorded by the source corpus.

We argue that representations in AI models, particularly deep networks, are converging. First, we survey many examples of convergence in the literature: over time and across multiple domains, the ways by which different neural networks represent data are becoming more aligned. Next, we demonstrate convergence across data modalities: as vision models and language models get larger, they measure distance between datapoints in a more and more alike way. We hypothesize that this convergence is driving toward a shared statistical model of reality, akin to Plato's concept of an ideal reality. We term such a representation the platonic representation and discuss several possible selective pressures toward it. Finally, we discuss the implications of these trends, their limitations, and counterexamples to our analysis.

Subfields: Philosophy

View Paper: https://www.semanticscholar.org/paper/66de49b3dcbbf0cca535335d597f94b702e2b95a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think (2024, positive): “We also include linear probing results (Acc.) and CKNNA (Huh et al., 2024) as discussed in Section 3.2.”
The TellTail of Embeddings: Fingerprinting Retrievers in Black-Box Systems (2026, unknown): “Identifying them may even seem unlikely, as prior work has argued that embedding models converge toward a shared geometry [14], [15], often retrieving overlapping passages for queries [34].”
Jailbreak susceptibility prediction and mitigation via the behavioral geometry of models (2026, unknown): “Low-dimensional representations of language models have been constructed from internal activations [33, 22, 18, 10], model weights [2], and embeddings of model responses [16, 14, 1, 5].”
Control Charts for Multi-agent Systems (2026, unknown): “Prior work has embedded models via internal activations [8, 13], parameter weights [20], or responses to reference queries [1].”
Black-box model classification under the discriminative factorization (2026, unknown): “In particular, if modern models are converging toward shared representations [17, 20], then variation introduced by a specific modification–fine-tuning on particular data, inserting a system prompt, or connecting a document store – perturbs the base representation along a few task-relevant…”
Better Language Models Exhibit Higher Visual Alignment (2024, unknown): “In contrast, Sharma et al. (2024) and Huh et al. (2024) more immediately assess the visual nature of LLMs and highlight a non-trivial degree of visual understanding and cross-modal alignment.”