Clustering with Bregman Divergences

By Arindam Banerjee, Srujana Merugu, Inderjit S. Dhillon, Joydeep Ghosh. 2005

1,940 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistics and Probability, Algorithms & Complexity

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistics and Probability

In Algorithms & Complexity

Citations in the corpus

Prototypical Networks for Few-shot Learning (2017, methodology): “For a particular class of distance functions, known as regular Bregman divergences [4], the Prototypical Networks algorithm is equivalent to performing mixture density estimation on the support set with an exponential family density.”
Three Approaches for Personalization with Applications to Federated Learning (2020, methodology): “If there are no meta-features, a natural approach is to cluster using a Bregman divergence deﬁned over the distributions D k (Banerjee et al., 2005).”
Adaptive Gradient-Based Meta-Learning Methods (2019, background): “An everywhere sub-differentiable function f : Finally, we will also consider functions that are exp-concave [29]: We now turn to the Bregman divergence and a discussion of several useful properties [10, 6]: Deﬁnition A.4.”
Agnostic Federated Learning (2019, methodology): “Thus, techniques such as those of Banerjee, Merugu, Dhillon, and Ghosh (2005) could be used to determine clusters of clients using a suitable Bregman divergence.”
Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams (2020, background): “This assumption is optimal for Bregman divergences for which the cluster mean is shown to have minimal distance to its population [5].”
Reward Augmented Maximum Likelihood for Neural Structured Prediction (2016, background): “A Bregman divergence is defined by a strictly convex, differentiable, closed potential function F : F → R [3].”
The Utility of Sparse Representations for Control in Reinforcement Learning (2018, background): “For exponential families, the KL divergence correspond to a Bregman divergence (Banerjee et al. 2005): for a convex potential function F that depends on the exponential family.”
Learning with Submodular Functions: A Convex Optimization Perspective (2011, neutral): “(wt−1). We denote by DΨ the Bregman divergence associated with the strongly convex function Ψ, i.e., DΨ(v,w) = Ψ(v) − Ψ(w) − (v− w)⊤Ψ′(w). See Figure 7.3 for an illustration and further properties in [14]. The previous iteration may be seen as the one of minimizing with respect to wa certain functio”
Simplifying Mixture Models Through Function Approximation (2006, background): “The comparison of the time consumptions can be found in Section V-C.”
Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation (2014, methodology): “An example of a family of convex loss functions is Bregman divergence [1], which includes a variety of loss functions such as squared loss, logistic loss, ItakuraSaito distance, squared Euclidean distance, Mahalanobis distance, KL-divergence and generalized I-divergence.”
Bregman Alternating Direction Method of Multipliers (2013, methodology): “More details about Bregman divergence can be found in [6, 1].”
A classification for community discovery methods in complex networks (2011, neutral): “However, the framework is presented here because it is possible to apply its principles to all the other community discovery algorithms presented in this survey.”
A Unified and General Framework for Continual Learning (2024, methodology): “The Bregman divergence (Banerjee et al., 2005) related to Φ for two points p and q within the set Ω can be understood as the discrepancy between the Φ value at point p and the value obtained by approximating Φ using ﬁrst-order Taylor expansion at q .”
Curvilinear component analysis and Bregman divergences (2010, background): “A Bregman divergence [1] between two points, p and q ∈ S, is defined to be dF (p,q) = F (p)− F (q)− 〈(p− q),∇F (q)〉 (6) where the angled brackets indicate an inner product and ∇F (q) is the derivative of F evaluated at q.”
TapNet: Multivariate Time Series Classification with Attentional Prototypical Network (2020, background): “The distance function can be chosen from regular Bregman divergences (Banerjee et al. 2005).”
Relational learning via collective matrix factorization (2008, background): “Our primary focus is on decomposable regular Bregman divergences [6], which correspond to maximum likelihood in exponential families:”
Hypothesis Testing, Information Divergence and Computational Geometry (2013, background): “Those dual coordinate systems are related by the Legendre transformation [6] F ∗(η) = supθ∈Θ θ >η−F (θ).”