Exploiting Generative Models in Discriminative Classifiers

By Tommi S. Jaakkola, David Haussler. 1998

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

Citations in the corpus

Aggregating Local Image Descriptors into Compact Codes (2012, background): “Jaakkola and Haussler [20] proposed to describe X by the vector : The gradient of the log-likelihood describes the contribution of the parameters to the generation process.”
Discriminative Embeddings of Latent Variable Models for Structured Data (2016, neutral): “We compare our algorithm on string benchmark datasets with the kernel method with existing sequence kernels, i.e., the spectrum string kernel (Leslie et al., 2002a), mismatch string kernel (Leslie et al., 2002b) and fisher kernel with HMM generative models (Jaakkola & Haussler, 1999).”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “In Cuturi et al. (2005), the semi-group kernels were designed for objects with additive semi-group structure such 10Intuitively, the gradient of log-likelihood ui specifies how the parameter θ contributes to the process of generating the example xi (Jaakkola and Haussler 1998).”
Distribution-Free Distribution Regression (2013, background): “[2004], Jaakkola and Haussler [1998]), usually fit a parametric family (e.”
Generative and Latent Mean Map Kernels (2010, methodology): “We similarly define p ′, q′, and x′ for a second HMM. Suppose that we have learned HMMs with probability measures p and p′ and wish to compute the GMMK km(p, p ′) for the observable variables of length T sequences drawn from these HMMs.”
Model Zoo: A Growing Brain That Learns Continually (2021, background): “…e.g., (Evgeniou et al., 2005; Cavallanti et al., 2010; Kumar and Daume III, 2012), and although such methods are popular in transfer learning (Pentina and Lampert, 2015; Jaakkola and Haussler, 1999), one cannot apply them in continual learning because we do not know the tasks beforehand.”
Structured Generative Models of Natural Source Code (2014, background): “Our hope is that by building better generative models, we can make progress simultaneously towards several of the above-mentioned tasks, and that we can better isolate and tackle the core challenges that arise when applying machine learning methods to source code.”
Bag of visual words and fusion methods for action recognition: Comprehensive study and good practice (2014, neutral): “Fisher vector is another super vector based encoding method derived from fisher kernel [15] and is introduced for large-scale image categorization [35].”
Task2Vec: Task Embedding for Meta-Learning (2019, methodology): “Fisher kernels Our work takes inspiration from Jaakkola and Hausler [16].”
Propagation kernels: efficient graph kernels from propagated information (2014, methodology): “Propagation kernels measure the similarity of two graphs by comparing node label or attribute distributions after each step of an appropriate random walk.”
Scalable Multitask Representation Learning for Scene Classification (2014, result): “Recent work [15] showed that feature encoding based on the Fisher Kernel [8], which combines the benefits of generative and discriminative approaches, yields significantly better results when compared to the standard bag of visual (BoV) words model on a number of computer vision benchmarks.”
Fisher Vector Faces in the Wild (2013, methodology): “the Gaussian Mixture Model (GMM), to the features, and then encoding the derivatives of the log-likelihood of the model with respect to its parameters [14].”
Machine Learning Paradigms for Speech Recognition: An Overview (2013, methodology): “The work of [131] makes use of the Fisher kernel to exploit generative models in discriminative classifiers.”
Deep learning for visual understanding (2017, unknown): “[147] overcame this issue and extracted higher order statistics by employing the Fisher Kernel [148], achieving the stateof-the-art image classi cation result in 2011.”
Encoding High Dimensional Local Features by Sparse Coding Based Fisher Vectors (2014, unknown): “Fisher vector coding is a coding method derived from the Fisher kernel [1] which was originally proposed to compare two samples induced by a generative model.”
ACTIVE: Activity Concept Transitions in Video Event Classification (2013, methodology): “The idea of Fisher kernel is first proposed in [7], the goal is to get the sufficient statistics of a generative model, and use them as kernel functions in discriminative classifiers such as SVM.”
Linear-Time Learning on Distributions with Approximate Kernel Embeddings (2015, background): “Another approach estimates a distance between distributions, often the L 2 distance or Kullback-Leibler ( KL ) divergence, parametrically (Jaakkola and Haussler 1998; Moreno, Ho, and Vasconcelos 2003; Jebara, Kondor, and Howard 2004) or nonparametrically (Sricharan, Wei, and Hero 2013;…”