On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes

By Andrew Y. Ng, Michael I. Jordan. 2001

2,649 citations recorded by the source corpus.

Subfields: Model Architectures, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

In Evaluation & Benchmarks

Citations in the corpus

Generative adversarial networks (2014, neutral): “If your task is to determine P(Y | X ), then why would you want to model P(X ,Y ) as an intermediate step? Ng and Jordan (2001) shows that generative models can be useful even for traditionally discriminative problems.”
Do Not Have Enough Data? Deep Learning to the Rescue! (2019, methodology): “For word embedding, we employed GLoVe (Pennington, Socher, and Manning 2014) of 100 dimensions.”
Meta-Learning Probabilistic Inference for Prediction (2018, neutral): “First, the use of discriminative models to maximize predictive performance on supervised learning tasks (Ng and Jordan, 2002).”
Invariant Scattering Convolution Networks (2012, background): “Although discriminant classiﬁers such as SVM have better asymptotic properties than generative classiﬁers [26], the situation can be inverted for small training sets.”
Posterior calibration and exploratory analysis for natural language processing models (2015, background): “Since logistic regression has the same log-linear representational capacity (Ng and Jordan, 2002) but does not suffer from the independence assumptions, we select it for comparison, hypothesizing it may have better calibration.”
Generative Modeling of Convolutional Neural Networks (2014, background): “The relationship between the generative models and the discriminative approaches has been extensively studied, perhaps starting from Efron (Efron, 1975), and more recently by (Jordan, 2002; Liang & Jordan, 2008), etc.”
Discriminative Training of a Neural Network Statistical Parser (2004, methodology): “In this form, the distinction between our two models is sometimes referred to as \joint versus conditional" (Johnson, 2001; Klein and Manning, 2002) rather than \generative versus discriminative" (Ng and Jordan, 2002).”
Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning (2020, background): “Ng & Jordan (2001) show that classiﬁers trained with generative losses can outperform their counterparts trained with discriminative losses in the context of Logistic Regression and Naive Bayes.”
Answer-Type Prediction for Visual Question Answering (2016, result): “This is consistent with earlier observations [13], although it is somewhat surprising because the covariance matrices used in our Bayesian model require a significant amount of data to accurately estimate.”
Generative versus discriminative training of RBMs for classification of fMRI images (2008, background): “In regimes in which the number of training examples is relatively small, it has been shown that classifiers based on generative models can outperform discriminative classifiers, e.g., naive Bayes classifiers can beat logistic regression [11].”
A Multivariate Approach for Patient-Specific EEG Seizure Detection Using Empirical Wavelet Transform (2017, background): “5 [43], functional tree (FT) [44], Bayes net [45], Naive-Bayes [46], and K-nearest neighbors (K-NN) [47] classifiers that are available at Waikato environment for knowledge analysis software [48].”
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization (2021, background): “Generative Models for Image Understanding: Our approach is in line with various works that explore the use of generative modeling in different forms for discriminative and image recognition tasks, an idea that dates back until at least [65] and has also been studied using early energybased models [7”
Convex Discriminative Multitask Clustering (2013, methodology): “This phenomenon has been observed in both the supervised classification [26] and the clustering [27].”
Triple Generative Adversarial Networks (2019, methodology): “In contrast, generative models in general [10] or the recent development of deep generative models (DGMs) [13], [14], [15], [16] provide a complementary solution.”