Principled Hybrids of Generative and Discriminative Models

By Julia A. Lasserre, Charles M. Bishop, Tom Minka. 2006

392 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Vector-quantized Image Modeling with Improved VQGAN (2021, background): “(Lasserre et al., 2006); importantly, there are no limitations on whether the signals are from the language domain or others, such as vision.”
MixMatch: A Holistic Approach to Semi-Supervised Learning (2019, neutral): “, “transductive” models [14, 22, 21], graph-based methods [49, 4, 29], generative modeling [3, 27, 41, 9, 17, 23, 38, 34, 42], etc.”
GeDi: Generative Discriminator Guided Sequence Generation (2020, methodology): “nces in the desired class. The idea of discriminatively training class-conditional generative models has previously been considered for the classiﬁcation of text (Yakhnenko et al., 2005), and images (Lasserre et al., 2006). With GeDi training, we combine the standard generative language modeling los”
A note on the evaluation of generative models (2015, background): “…, and parameters θ , the only correct way to infer the distribution over y —from a Bayesian point of view —is to integrate out the parameters (e.g., Lasserre et al., 2006), With sufﬁcient data and under certain assumptions, the above integral will be close to p ( y | x , ˆ θ MAP ) , When no…”
Generative replay with feedback connections as a general strategy for continual learning (2018, background): “Besides removing the need for a separate generative model, it is possible that regularization provided by the added generative objective helps to train a more robust classiﬁer [23, 24].”
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks (2018, background): “This classiﬁer has been studied in various machine learning areas (e.g., semi-supervised learning [17] and incremental learning [29]).”
Adversarial Dropout for Supervised and Semi-supervised Learning (2017, methodology): “Several techniques (Hinton et al. 2012; Poole, Sohl-Dickstein, and Ganguli 2014; Bishop 1995b; Lasserre, Bishop, and Minka 2006) have emerged over the past years to address this challenge, and dropout has become one of dominant methods due to its simplicity and effectiveness (Hinton et al. 2012;…”
Smooth Neighbors on Teacher Graphs for Semi-Supervised Learning (2017, background): “Apart from discriminative approaches, another line is generative models, which pay efforts to learn the input distribution p ( x ) that is believed to share some information with the conditional distribution p ( y | x ) [22].”
Recent Advances in Open Set Recognition: A Survey (2018, background): “There is an ongoing debate between the use of generative and discriminative models in statistical learning [65], [66], with arguments for the value of each.”
Hybrid Models with Deep and Invertible Features (2019, methodology): “Weighted losses are commonly used in hybrid models (Lasserre et al., 2006; McCallum et al., 2006; Kingma et al., 2014; Tulyakov et al., 2017).”
Robust Inference via Generative Classifiers for Handling Noisy Labels (2019, background): “This classiﬁer has been studied in various machine learning areas (e.g., semi-supervised learning (Lasserre et al., 2006) and incremental learning (Lee et al., 2018)).”
Learning Generative Models via Discriminative Approaches (2007, background): “The purpose of hybrid models in [14] is to study different priors over parameters.”
Realistic Evaluation of Semi-Supervised Learning Algorithms (2018, methodology): “…2003; Bengio et al., 2006), and algorithms based on learning features (frequently via generative modeling) from unlabeled data (Belkin & Niyogi, 2002; Lasserre et al., 2006; Salakhutdinov & Hinton, 2007; Coates & Ng, 2011; Goodfellow et al., 2011; Kingma et al., 2014; Pu et al., 2016; Odena, 2016;…”
Why Does Unsupervised Pre-training Help Deep Learning? (2010, background): “It can then be seen that the generative criterion corresponds to a particular form of prior (Lasserre et al., 2006), namely that the structure of P(X) is connected to the structure of P(Y |X) in a way that is captured by the shared parametrization.”
Asymptotically Optimal Regularization in Smooth Parametric Models (2009, methodology): “We might regularize with a generative model if we think it is roughly well-specified [7, 20, 15, 17].”
Generative Pretraining From Pixels (2020, background): “It was believed that a model which learned the data distribution P (X) would also learn beneficial features for the subsequent supervised modeling of P (Y |X) (Lasserre et al., 2006; Erhan et al., 2010).”
Data Augmentation via Levy Processes (2016, background): “A number of papers study hybrid loss functions that combine both a joint and conditional likelihood (Raina et al., 2004; Bouchard and Triggs, 2004; Lasserre et al., 2006; McCallum et al., 2006; Liang and Jordan, 2008).”
Learning Visual Object Categories for Robot Affordance Prediction (2010, methodology): “A similar approach was described by Lasserre et al. (2006), who point out the difficulty of optimizing CLL using EM, and instead adopt a conjugate gradient method to directly update the parameters.”