Learning from Distributions via Support Measure Machines

By Krikamol Muandet, Kenji Fukumizu, Francesco Dinuzzo, Bernhard Schölkopf. 2012

203 citations recorded by the source corpus.

This paper presents a kernel-based discriminative learning framework on probability measures. Rather than relying on large collections of vectorial training examples, our framework learns using a collection of probability distributions that have been constructed to meaningfully represent training data. By representing these probability distributions as mean embeddings in the reproducing kernel Hilbert space (RKHS), we are able to apply many standard kernel-based learning techniques in straightforward fashion. To accomplish this, we construct a generalization of the support vector machine (SVM) called a support measure machine (SMM). Our analyses of SMMs provides several insights into their relationship to traditional SVMs. Based on such insights, we propose a flexible SVM (Flex-SVM) that places different kernel functions on each training example. Experimental results on both synthetic and real-world data demonstrate the effectiveness of our proposed framework.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Probabilistic & Bayesian Methods

Citations in the corpus

Deep Perm-Set Net: Learn to predict sets with unknown permutation and cardinality using deep neural networks (2018, methodology): “Approaches such as mixture models Blei et al. (2003); Hannah et al. (2011); Tran et al. (2016), learning distributions from a set of samples Muandet et al. (2012); Oliva et al. (2013), model-based multiple instance learning Vo et al. (2017) and novelty detection from point pattern data Vo et al.…”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “A slightly different interpretation is to look at this problem as a learning problem on probability distributions (Muandet et al. 2012, Muandet and Schölkopf 2013, Szabó et al. 2015).”
Set Transformer (2018, background): “…shape recognition (Wu et al., 2015; Shi et al., 2015; Su et al., 2015; Charles et al., 2017), sequence ordering (Vinyals et al., 2016), and various set operations (Muandet et al., 2012; Oliva et al., 2013; Edwards & Storkey, 2017; Zaheer et al., 2017) can also be viewed as the set-input problems.”
Prototypal Analysis and Prototypal Regression (2017, background): “In prior work, Muandet et al. (2012) extends support vector machine to support measure machine for classification of distributions using the kernel embedding induced inner product.”
Computing functions of random variables via reproducing kernel Hilbert space representations (2015, neutral): “2013), and supervised learning on probability distributions (Muandet et al. 2012; Szabó et al. 2014).”
Non-Linear Regression for Bag-of-Words Data via Gaussian Process Latent Variable Set Model (2015, methodology): “For example, the support measure machine is a method for kernel-based discriminative learning on distributions, which generalizes the support vector machine by kernel embeddings (Muandet et al. 2012).”
Characteristic Kernels and Infinitely Divisible Distributions (2014, background): “, k(P, Pθ) = exp(− ||mP−mPθ || 2 H 2σ2 ) need to be computed, as in the support measure machine (Muandet et al., 2012).”
Fast Distribution To Real Regression (2013, background): “Futhermore, the representer theorem was recently generalized for the space of distributions [11].”
Distribution Kernel Methods for Multiple-Instance Learning (2013, methodology): “Recent techniques for learning from distributions with kernels (Smola et al. 2007; Muandet et al. 2012) have the potential to be successfully applied to these domains and other MI learning problems.”
Kernel Mean Estimation and Stein's Effect (2013, positive): “We consider three benchmark applications: density estimation via kernel mean matching [20], kernel PCA using shrinkage mean and covariance operator [2], and discriminative learning on distributions [25, 26].”
One-Class Support Measure Machines for Group Anomaly Detection (2013, background): “This kernel is particularly useful when one want to incorporate the point-wise uncertainty of the observation into the learning algorithm (Muandet et al. 2012).”
Distribution-Free Distribution Regression (2013, background): “Recently, the representer theorem has also been generalized for the space of probability distributions [Muandet et al., 2012].”
Domain Generalization via Invariant Feature Representation (2013, background): “Here, k X is the kernel on H X and the kernel on distributions is k P ( P , P ′ ) := κ ( µ P , µ P ′ ) where κ is a positive deﬁnite kernel (Christmann and Steinwart 2010, Muandet et al. 2012).”
Deep Sets (2017, background): “…however, can only deal with ﬁxed, ﬁnite-dimensional representations, and researchers only recently started to investigate the case when the inputs or outputs are permutation invariant sets rather than ﬁnite dimensional vectors (Oliva et al., 2013; Szabo et al., 2016; Muandet et al., 2013; 2012).”
Meta-Curvature (2019, neutral): “4 of [28]).”