Kernel independent component analysis

By Francis R. Bach, Michael I. Jordan. 2003

2,000 citations recorded by the source corpus.

Subfields: Analytical Chemistry, Artificial Intelligence, Representation & Self-Supervised Learning, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Analytical Chemistry

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Signal Processing

Citations in the corpus

A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation (2020, methodology): “In a similar spirit to disentanglement, (non-)linear independent component analysis (Comon, 1994; Bach and Jordan, 2002; Jutten and Karhunen, 2003; Hyvarinen and Morioka, 2016) studies the problem of recovering independent components of a signal.”
Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics (2013, methodology): “Most of the systems we present are based on a technique called Kernel Canonical Correlation Analysis (Bach & Jordan, 2002; Hardoon et al., 2004).”
Beyond CCA: Moment Matching for Multi-View Models (2016, positive): “Importantly, the joint diagonalization type algorithms are known to be more stable in practice (see, e.g., Bach & Jordan, 2003; Podosinnikova et al., 2015).”
Rethinking LDA: Moment Matching for Discrete ICA (2015, critical): “In practice, the orthogonal joint diagonalization (JD) algorithm is more robust than FastICA (see, e.g., [26, p. 30]) or the spectral algorithm.”
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics (2012, methodology): “However, we will formulate KCCA as solving for a linear projection from the kernel space, because this leads directly to our scalable approximation scheme based on explicit embeddings.”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “The covariance operator serves as a basic building block in classical kernel-based methods such as kernel PCA (Schölkopf et al. 1998, Zwald et al. 2004), kernel Fisher discriminant, kernel CCA (Fukumizu et al. 2007), and kernel ICA (Bach and Jordan 2003).”
Flexibly Fair Representation Learning by Disentanglement (2019, background): “The search of independent latent components that explain observed data has long been a focus on the probabilistic modeling community (Comon, 1994; Hyv¨arinen & Oja, 2000; Bach & Jordan, 2002).”
On the Transfer of Disentangled Representations in Realistic Settings (2020, background): “Ideas related to disentangling the factors of variation date back to the non-linear ICA literature (Comon, 1994; Hyv¨arinen and Pajunen, 1999; Bach and Jordan, 2002; Jutten and Karhunen, 2003; Hyvarinen and Morioka, 2016; Hyvarinen et al., 2019; Gresele et al., 2019).”
On the Fairness of Disentangled Representations (2019, background): “Ideas related to disentangling the factors of variations have a long tradition in machine learning, dating back to the non-linear ICA literature [17, 3, 46, 42, 43, 44, 32].”
Are Disentangled Representations Helpful for Abstract Visual Reasoning? (2019, background): “Other Related Works Learning disentangled representations is similar in spirit to non-linear ICA, although it relies primarily on (architectural) inductive biases and different degrees of supervision [13, 2, 39, 36, 37, 38, 25, 33, 32].”
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations (2018, methodology): “In a similar spirit to disentanglement, (non-)linear independent component analysis (Comon, 1994; Bach & Jordan, 2002; Jutten & Karhunen, 2003; Hyvarinen & Morioka, 2016) studies the problem of recovering independent components of a signal.”
Computing functions of random variables via reproducing kernel Hilbert space representations (2015, neutral): “In the latter case,we estimate ∥μ[px py]− μ[pxy] ∥∥ (Bach and Jordan 2002; Gretton et al. 2005); in the former case, we estimate ∥μ[p] − μ[p′]∥ (Gretton et al.”
Filtering with State-Observation Examples via Kernel Monte Carlo Filter (2013, methodology): “approximation works well in practice, since eigenvalues of a kernel matrix often decay very rapidly. Indeed this has been theoretically shown for some cases; see Widom (1963, 1964) and discussions in Bach and Jordan (2002). (ii) A data reduction method based on Kernel Herding, which efﬁciently selec”
A Kernel Test for Three-Variable Interactions (2013, background): “Much of the work in this area focuses on measuring or testing pairwise interaction: for instance, the Hilbert-Schmidt Independence Criterion (HSIC) or Distance Covariance [1, 2, 3], kernel canonical correlation [4, 5, 6], and mutual information [7].”
Equivalence of distance-based and RKHS-based statistics in hypothesis testing (2012, methodology): “(2008): we generated univariate random variables from the Independent Component Analysis (ICA) benchmark densities of Bach and Jordan (2002); rotated them in the product space by an angle between 0 and π/4 to introduce dependence; filled additional dimensions with independent Gaussian noise; and, fi”
Hypothesis testing using pairwise distances and associated kernels (2012, methodology): “…we used an artificial benchmark proposed by Gretton et al. (2008): we generate univariate random variables from the ICA benchmark densities of Bach & Jordan (2002); rotate them in the product space by an angle between 0 and π/4 to introduce dependence; fill additional dimensions with…”
Kernel Belief Propagation (2011, positive): “In many cases of interest (for instance, when a Gaussian RBF kernel is used), a small ` m will be sufficient to obtain a small residual for the feature matrix, due to the fast decay of the eigenspectrum in feature space (Bach & Jordan, 2002, Appendix C).”
Hilbert Space Embeddings and Metrics on Probability Measures (2009, methodology): “Keywords: Probability metrics, Homogeneity tests, Independence tests, Kernel methods, Universal kernels, Characteristic kernels, Hilbertian metric, Weak topology.”
A Kernel Method for the Two-Sample-Problem (2006, methodology): “Kernel methods were employed to solve this problem by Bach and Jordan (2002); Gretton et al.”