Exact Recovery in the Stochastic Block Model

By Emmanuel Abbe, Afonso S. Bandeira, Georgina Hall. IEEE Transactions on Information Theory, 2014

615 citations recorded by the source corpus.

The stochastic block model with two communities, or equivalently the planted bisection model, is a popular model of random graph exhibiting a cluster behavior. In the symmetric case, the graph has two equally sized clusters and vertices connect with probability p within clusters and q across clusters. In the past two decades, a large body of literature in statistics and computer science has focused on providing lower bounds on the scaling of | p - q| to ensure exact recovery. In this paper, we identify a sharp threshold phenomenon for exact recovery: if α = pn/log(n) and β = qn/ log(n) are constant (with α > β), recovering the communities with high probability is possible if (α + β/2) - √(αβ) > 1 and is impossible if (α + β/2) - √(αβ) <; 1. In particular, this improves the existing bounds. This also sets a new line of sight for efficient clustering algorithms. While maximum likelihood (ML) achieves the optimal threshold (by definition), it is in the worst case NP-hard. This paper proposes an efficient algorithm based on a semidefinite programming relaxation of ML, which is proved to succeed in recovering the communities close to the threshold, while numerical experiments suggest that it may achieve the threshold. An efficient algorithm that succeeds all the way down to the threshold is also obtained using a partial recovery algorithm combined with a local improvement procedure.

View Paper: https://www.semanticscholar.org/paper/f0afbf9ca17b31fa6d59bb524184756c06a5e050

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks (2019, background): “3,48] for additional work on dealing with label noise in classication tasks. When learning from pairwise relations, noisy labels can be connected to graph clustering and community detection problems [1,14,54]. Label noise is also connected to outlier robustness in regression which is a traditionally”
Random Growth Models (2018, background): “I(ρ,N) N/ logN ≤ n ( 2 π + o(1))n = π 2 + o(1).”
Community Detection with Graph Neural Networks (2017, background): “In the two community case, It was shown that exact recovery is possible on SBM ( n, p = √ et al., 2014; Abbe et al., 2014).”
Statistical Inference on Random Dot Product Graphs: a Survey (2017, methodology): “In the study of stochastic blockmodels, there has been a recent push to understand the fundamental information-theoretic limits for community detection and graph partitioning [1, 2, 71, 72].”
Supervised Community Detection with Line Graph Neural Networks (2017, background): “In the two community case, It was shown that exact recovery is possible on SBM(n, p = a lognn , q = b logn n ) if and only if a+b 2 ≥ 1 + √ ab (Mossel et al., 2014; Abbe et al., 2014).”
Community Detection in General Stochastic Block models: Fundamental Limits and Efficient Algorithms for Recovery (2015, methodology): “for weak recovery [37], [38] and exact recovery [39], [40].”