Density-ratio matching under the Bregman divergence: a unified framework of density-ratio estimation

By Masashi Sugiyama, Teruyuki Suzuki, Takafumi Kanamori. 2012

228 citations recorded by the source corpus.

Subfields: Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, methodology): “The FactorVAE (Kim and Mnih, 2018) and the β-TCVAE (Chen et al., 2018) penalize the total correlation (Watanabe, 1960) with adversarial training (Nguyen et al., 2010; Sugiyama et al., 2012) or with a tractable but biased Monte-Carlo estimator respectively.”
Disentangling by Factorising (2018, methodology): “Having access to samples from both distributions allows us to minimise their KL divergence using the density-ratio trick (Nguyen et al., 2010; Sugiyama et al., 2012) which involves training a classifier/discriminator to approximate the density ratio that arises in the KL term.”
Recent Advances in Autoencoder-Based Representation Learning (2018, methodology): “To estimate it from samples, Kim and Mnih [3] rely on the density ratio trick [32, 33] which involves training a discriminator (see Section 2).”
Resampled Priors for Variational Autoencoders (2018, background): “Density Ratio Estimation: DRE (Sugiyama et al., 2012) estimates a ratio of densities by training a classifier to discriminate between samples from them.”
Deep and Hierarchical Implicit Models (2017, methodology): “We apply techniques for ratio estimation (Sugiyama et al., 2012).”
Hierarchical Implicit Models and Likelihood-Free Variational Inference (2017, methodology): “For inference, § 3 develops likelihood-free variational inference (LFVI), which combines variational inference with density ratio estimation [49, 35].”
Kernel Implicit Variational Inference (2017, methodology): “We also refer the readers to many other works by Sugiyama and his collaborators on DRE, such as KLIEP (Sugiyama et al., 2008), LSIF (Kanamori et al., 2009), and DRE based on Bregman divergence (Sugiyama et al., 2012).”
Semi-Supervised Learning of Class Balance under Class-Prior Change by Distribution Matching (2012, background): “is more useful for our purpose of learning class ratios, because of the following reasons: The PE-divergence was shown to be more robust against outliers than the KLdivergence, based on power divergence analysis (Basu et al., 1998; Sugiyama et al., 2012).”
Under Review as a Conference Paper at Iclr 2017 Learning in Implicit Generative Models (2017, methodology): “There are four general approaches to consider (Sugiyama et al., 2012a): 1) class-probability estimation, 2) divergence minimisation, 3) ratio matching, and 4) moment matching.”
Dynamic Likelihood-free Inference via Ratio Estimation (DIRE) (2018, methodology): “…This is exploited by Dutta et al. (2016) who estimate the ratio r(θ,x) by training a logistic regression model to learn to classify between data sampled from p(x | θ) and p(x) (other methods to estimate the ratio can also be used, see Gutmann and Hirayama 2011; Sugiyama et al. 2012).”
A Review of Domain Adaptation without Target Labels (2019, methodology): “A simple procedure is to use logistic regression to discriminate between samples from each domain [105].”
Gradient Estimators for Implicit Models (2017, methodology): “Related work for wild variational inference [19, 26, 13, 43] uses a GAN-based technique to construct a density ratio estimator for q/p0 [41, 42, 45, 27] and then approximate the KL-divergence term in the variational lower-bound: LVI(q) = Eq [log p(x|z)]−KL[qφ(z|x)||p0(z)].”
LTF: A Label Transformation Framework for Correcting Label Shift (2020, background): “The matching of distributions is achieved by minimizing suitable divergence measures (Gretton et al., 2012; Sugiyama et al., 2012) w.”
Augmented Variational Autoencoders for Collaborative Filtering with Auxiliary Information (2017, background): “Recent literature on statistics [25, 37] has identied that the formulation of GAN can be interpreted as the problem of learning implicit generative models using the discriminator as a proxy of the density ratio r = pdata pθ .”
Clustering via Mode Seeking by Direct Estimation of the Gradient of a Log-Density (2014, background): “On the other hand, f(t) = − log t gives the Kullback-Leibler loss [17], f(t) = t log t− (1 + t) log(1 + t) gives the logistic loss [23], and f(t) = (t − t)/α for α > 0 gives the power loss [2].”
Wild Variational Approximations (2016, methodology): “A new direction for energy approximation applies density ratio estimation methods [33, 34, 35].”
Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation (2012, methodology): “Moreover, it has the optimal numerical stability (Kanamori et al., 2013), and it is more robust than KLIEP (Sugiyama et al., 2012a).”