Deterministic annealing for clustering, compression, classification, regression, and related optimization problems

By Kenneth M. Rose. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Learning How to Ask: Querying LMs with Mixtures of Soft Prompts (2021, methodology): “We used only a single copy of each prompt, but a generalization would be to allow multiple slightly perturbed copies of each prompt, which could di-verge and specialize during training (Rose, 1998).”
Regularizing Neural Networks by Penalizing Confident Output Distributions (2017, methodology): “Deterministic annealing Rose (1998) is a general approach for optimization that is widely applicable, avoids local minima, and can minimize discrete objectives, and it can be derived from the maximum entropy principle.”
Deep learning and the information bottleneck principle (2015, methodology): “Following the bifurcation analysis of the cluster splits in [14], [12] for the IB phase transitions, one can show that the critical β is determined by the largest eigenvalue of the second order correlations of p(x, y|x̂(β)), at that critical β.”
A Distributed Deterministic Annealing Algorithm for Limited-Range Sensor Coverage (2011, methodology): “Of these techniques, there are simulated annealing (SA) algorithms [7], as well as a more recent development, deterministic annealing (DA) [ 8 ].”
Graduated Non-Convexity for Robust Spatial Perception: From Non-Minimal Solvers to Global Outlier Rejection (2019, methodology): “Graduated non-convexity (GNC) is a popular approach for the optimization of a generic non-convex cost function ρ(·) and has been used in several endeavors, including vision [33] and machine learning [45] (see [46] for more applications).”
A Unified Framework for Model-based Clustering (2003, methodology): “At a high temperature, the objective function (Equation 4) is smooth and there are very few local solutions (Rose, 1998 ).”
Learning Structured Predictors from Bandit Feedback for Interactive NLP (2016, unknown): “Table 1: Metaparameter settings determined on dev sets for constant learning rate γt, temperature coefficient T0 for annealing under the schedule T = T0/ 3 √epoch + 1 (Rose, 1998; Arun et al., 2010), momentum coefficient min{1 − 1/(t/2 + 2), μ} (Polyak, 1964; Sutskever et al.”
Cluster analysis for gene expression data: a survey (2004, background): “[3] split the genes through a divisive approach, called the deterministic-annealing algorithm (DAA) [53], [52].”
Continuous latent variable models for dimensionality reduction and sequential data reconstruction (2001, unknown): “, 1977), to clustering by deterministic annealing (Rose, 1998) and to algorithms for finding pre-images in kernel-based methods (Schölkopf et al.”
Training with Confusion for Fine-Grained Visual Classification (2017, background): “Jaynes [13] introduced the maximum entropy principle, which provided an early understanding of the advantage of controlling the value of classifier entropy, leading to the work on deterministic entropy annealing [40].”
Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression (2007, background): “Readers who are interested only in the algorithm and experiments may bypass the next two sections and skip to Section V without any loss of continuity.”
Statistical shape analysis: clustering, learning, and testing (2005, background): “Several authors have explored the use of annealing in clustering problems, including soft clustering [22] and deterministic clustering [9].”