Estimating the number of clusters in a data set via the gap statistic

By Robert Tibshirani, Guenther Walther, Trevor J. Hastie. 2001

1,055 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Epidemiology

View Paper: https://www.semanticscholar.org/paper/e8020b85160eda580566053404cb5c32ad19c62c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Epidemiology

Citations in the corpus

Evaluation of Stability of k-Means Cluster Ensembles with Respect to Random Initialization (2006, positive): “Within the first approach, the indices by Calinski-Harabasz and Krzanowski-Lai have been repeatedly chosen as benchmarks [7], [25], [28], [ 33 ].”
Introduction to Information Retrieval (2008, neutral): “1 is inspired by a similar table in (van Rijsbergen 1979). Section 10.4 follows the overview of INEX 2002 by Gövert and Kazai (2003), published in the proceedings of the meeting (Fuhr et al.”
Flexible, Foldable, Actively Multiplexed, High-Density Electrode Array for Mapping Brain Activity in vivo (2011, neutral): “The potential number of clusters, k , ranged from 1 (that is, no clustering) to 30, and the gap statisti”
CLICK and EXPANDER: a system for clustering and visualizing gene expression data (2003, positive): “Another approach is to present a curve of homogeneity versus separation over a range of parameters for the clustering algorithm used (Ben-Dor, private communication).”
Combining Mixture Components for Clustering (2010, neutral): “An inferential choice could be made, for example using the gap statistic (Tibshirani, Walther, and Hastie 2001).”
Graph mining: Laws, generators, and algorithms (2006, neutral): “Also, the number of communities (e.g., the number of eigenvectors to be considered in spectral clustering) often needs to be set by the user, though some recent methods try to .nd this automatically [Tibshirani et al. 2001; Ben-Hur and Guyon 2003].”
An Evolutionary Approach to Multiobjective Clustering (2007, positive): “Tibshirani et al.’s Gap statistic [ 57 ], a statistical method to determine the number of clusters in a data set.”
Federated Adversarial Domain Adaptation (2019, positive): “Specifically, we leverage the gap statistics (Tibshirani et al., 2001) to evaluate how well the target features f t can be clustered with unsupervised clustering algorithms (K-Means).”
How Many Clusters? An Information-Theoretic Perspective (2003, neutral): “Such procedures include the definition of an intuitively reasonable criterion for the goodness of the classification, as in Tibshirani, Walther, and Hastie (2001) or performing cross-validation (Stone, 1974) and related methods in order to estimate the prediction error and to find the number of cl”
Participatory provenance as representational auditing for AI-mediated public consultation (2026, unknown): “…(ratio of between-cluster to within-cluster dispersion); (iii) the Davies-Bouldin index (mean ratio of intra-to inter-cluster scatter; lower is better); and (iv) the Gap Statistic [Tibshirani et al., 2001] (standardised comparison to a uniform reference distribution using 20 reference samples).”
ARTLAS: Mapping Art-Technology Institutions via Conceptual Axes, Text Embeddings, and Unsupervised Clustering (2026, unknown): “The gap statistic [31] independently yields 𝑘 ∗ = 8 with a plateau over 𝑘 ∈ [ 8 , 11 ] , confirming 𝑘 = 10 as statistically supported.”
Nemotron ColEmbed V2: Top-Performing Late Interaction embedding models for Visual Document Retrieval (2026, unknown): “We then apply PCA to reduce embeddings dimension to 50, followed by a K-Means clustering approach utilizing gap statistics [42] to choose the most representative k clusters.”