A Least-squares Approach to Direct Importance Estimation

By Takafumi Kanamori, Shohei Hido, Masashi Sugiyama. 2009

611 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

GenDICE: Generalized Offline Estimation of Stationary Values (2020, background): “Classical density ratio estimation includes moment matching (Gretton et al., 2008), probabilistic classification (Bickel et al., 2007), and ratio matching (Nguyen et al., 2008; Sugiyama et al., 2008; Kanamori et al., 2009).”
DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Corrections (2019, methodology): “error), various direct ratio estimators have been proposed [37], including the moment matching approach [13], probabilistic classification approach [3, 5, 33], and ratio matching approach [17, 27, 38] The proposed DualDICE algorithm, as a direct approach for density ratio estimation, bears some simi”
Watch and learn: Semi-supervised learning of object detectors from videos (2015, methodology): “Multiple feature spaces for false positive removal: We use the uLSIF [27] algorithm on Pyramidal HOG (PHOG) [4] and color histogram (LAB color with 32×16× 16 bins) features computed on a resized box of 200 × 300 px.”
Domain Generalization by Marginal Transfer Learning (2017, background): “For example, several works adopt the covariate shift assumption, which requires the source and target domains to have the same posterior, allowing the marginals to differ arbitrarily (Zadrozny, 2004; Huang et al., 2007; Cortes et al., 2008; Sugiyama et al., 2008; Bickel et al., 2009; Kanamori et al.”
Density-ratio matching under the Bregman divergence: a unified framework of density-ratio estimation (2012, methodology): “This is the solution of the approximation method called unconstrained LSIF (uLSIF; Kanamori et al. 2009).”
An introduction to domain adaptation and transfer learning (2018, background): “One could also minimize the squared error between the estimated weights and the actual ratio of distributions [116,115]:”
Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation (2012, methodology): “uLSIF has some notable advantages: Its solution can be computed analytically (Kanamori et al., 2009) and it possesses the optimal non-parametric convergence rate (Kanamori et al.”
Domain Adaptation by Joint Distribution Invariant Projections (2020, background): “(5) is a popular choice for approximating unknown target functions in statistical machine learning [56]–[58].”
A Bi-Hemisphere Domain Adversarial Neural Network Model for EEG Emotion Recognition (2021, methodology): “…including the methods of Kullback-Leibler importance estimation procedure (KLIEP) [41], unconstrained least-squares importance ﬁtting (ULSIF) [42], selective transfer machine (STM) [43], linear SVM [37], kernel principal component analysis (KPCA) [44], transfer component analysis (TCA) [45],…”
Conditional Density Estimation via Least-Squares Density Ratio Estimation (2010, methodology): “LS-CDE can be regarded as an application of the direct density-ratio estimation method called the unconstrained LeastSquares Importance Fitting (uLSIF) (Kanamori et al., 2009) to the problem of density ratio estimation.”
Constructive setting for problems of density ratio estimation (2015, methodology): “The optimization problem in (27) has a structure similar to that of the unconstrained Least-Squares Importance Fitting (uLSIF) method [9], where, instead of V′′ n×n and Vn× , one uses identity matrices In×n and In× , and, instead of regularization functional α K α, one uses α α.”
Likelihood-free inference with an improved cross-entropy estimator (2018, methodology): “This problem has prompted the development of likelihood-free inference methods such as Approximate Bayesian Computation [1–4] and neural density or neural density ratio estimation algorithms [5–23].”
Transfer String Kernel for Cross-Context DNA-Protein Binding Prediction (2016, background): “Several recent papers [25], [34], [35] aim to estimate the optimal weights by solving least-square based formulations.”
Implicit Variational Inference with Kernel Density Ratio Fitting (2017, methodology): “Based on the intuition, we apply a kernel density ratio fitting (KDRF) method similar to the unconstrained Least Square Importance Fitting (uLSIF) (Kanamori et al., 2009).”