Discriminative Clustering by Regularized Information Maximization

By Ryan Gomes, Andreas Krause, Pietro Perona. 2010

401 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets (2016, methodology): “More formally, the minimax game is given by the following expression: min G max D V (D,G) = Ex∼Pdata [logD(x)] + Ez∼noise[log (1−D(G(z)))] (1)”
On Mutual Information Maximization for Representation Learning (2019, background): “Already Linsker (1988) talks in his seminal work about constraints, while a manifestation of the problem in clustering approaches using MI criteria has been brought up by Bridle et al. (1992) and subsequently addressed using regularization by Krause et al. (2010).”
Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks (2015, methodology): “…times already in the literature (Bridle et al., 1992; Grandvalet & Bengio, 2005; Krause et al., 2010), and our objective function falls back to the regularized information maximization from Krause et al. (2010) when the generator is removed and the classiﬁer is additionally 2 regularized 6 .”
Unsupervised Learning by Predicting Noise (2017, methodology): “Many methods have been proposed to use discriminative losses for clustering (Xu et al., 2004; Bach & Harchaoui, 2007; Krause et al., 2010; Joulin & Bach, 2012).”
Third-Person Imitation Learning (2017, methodology): “In our optimization formulation, we apply the gradient ﬂipping technique from Ganin & Lempitsky (2014).”
On Variational Bounds of Mutual Information (2019, background): “As demonstrated in Krause et al. (2010), information maximization alone is ineffective at learning useful representations from ﬁnite data.”
Learning Discrete Representations via Information Maximizing Self-Augmented Training (2017, methodology): “nt. We can similarly impose the invariance via SAT by using the afﬁne distortion for the data augmentation. We then combine the SAT with the Regularized Information Maximization (RIM) for clustering (Gomes et al., 2010;Bridle et al.,1991), and arrive at our Information Maximizing Self-Augmented Trai”
Coordinated Multi-Agent Imitation Learning (2017, background): “This objective can also be seen as maximizing the mutual information between latent structure and observed trajectories (Krause et al., 2010).”
Putting An End to End-to-End: Gradient-Isolated Learning of Representations (2019, background): “constraint of temporal disparity. Thus, the InfoNCE loss can successfully enforce each module to preserve the information of its inputs, while providing the necessary regularization [Hu et al., 2017, Krause et al., 2010] for circumventing degenerate solutions. These factors contribute to ensuring th”
Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation (2020, methodology): “For this purpose, we adopt the information maximization (IM) loss (Krause et al., 2010; Shi & Sha, 2012; Hu et al., 2017), to make the target outputs individually certain and globally diverse.”
A Survey of Clustering With Deep Learning: From the Perspective of Network Architecture (2018, methodology): “It combines FCN and regularized Information Maximization (RIM) [50], which learns a probabilistic classifier such that mutual information between inputs and clus-”
Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation (2021, methodology): “We adopt the prediction diversity loss which is widely used in clustering [8, 9, 13] and also in several domain adaptation works [21, 39, 42]:”
Associative Deep Clustering: Training a Classification Network with No Labels (2018, methodology): “Information-Maximizing Self-Augmented Training (IMSAT) [18] is based on Regularized Information Maximization (RIM) [22], which learns a probabilistic classifier that maximizes the mutual information between the input and the class assignment.”
Pairwise Teacher-Student Network for Semi-Supervised Hashing (2019, background): “Various approaches are discovered such as transductive approach [9, 25], graphbased approach [1, 30], but they are not working well in complex dataset as the underlying structure of data is hard to capture.”
Source Data-Absent Unsupervised Domain Adaptation Through Hypothesis Transfer and Labeling Transfer (2020, methodology): “For this purpose, we adopt the information maximization (IM) loss [26], [27], [87] to make the classification outputs of target features individually certain and globally diverse.”
Toward a science of computational ethology. (2014, neutral): “by theoreticians to develop proof-of-principle experiments (Gomes et al., 2010).”
Subclass Distillation (2020, methodology): “The closest method to ours is that of Krause et al. (2010), which also uses a probabilistic classiﬁer for clustering by optimizing for class balance and class separation, although the authors use a different loss for this purpose, and perform experiments with kernel methods rather than deep neural…”
Unsupervised Joint Mining of Deep Features and Image Labels for Large-Scale Radiology Image Categorization and Scene Recognition (2017, methodology): “This advantage is probably due to RIM’s two unique properties: 1) less restricted geometric assumptions in the clustering feature space; 2) the capacity to attain the optimal number of clusters by maximizing the mutual information between input data and the induced clusters via a regularized term.”