Comparison of logistic regression and linear discriminant analysis

By Maja Pohar Perme, Mateja Blas, S. Turk. Advances in Methodology and Statistics, 2004

317 citations recorded by the source corpus.

Two of the most widely used statistical methods for analyzing categorical outcome variables are linear discriminant analysis and logistic regression. While both are appropriate for the development of linear classification models, linear discriminant analysis makes more assumptions about the underlying data. Hence, it is assumed that logistic regression is the more flexible and more robust method in case of violations of these assumptions. In this paper we consider the problem of choosing between the two methods, and set some guidelines for proper choice. The comparison between the methods is based on several measures of predictive accuracy. The performance of the methods is studied by simulations. We start with an example where all the assumptions of the linear discriminant analysis are satisfied and observe the impact of changes regarding the sample size, covariance matrix, Mahalanobis distance and direction of distance between group means. Next, we compare the robustness of the methods towards categorisation and non-normality of explanatory variables in a closely controlled way. We show that the results of LDA and LR are close whenever the normality assumptions are not too badly violated, and set some guidelines for recognizing these situations. We discuss the inappropriateness of LDA in all other cases.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/8f1834661a9dc854d09934e42ce4acf819d0b439

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Data mining methods in the prediction of Dementia: A real-data comparison of the accuracy, sensitivity and specificity of linear discriminant analysis, logistic regression, neural networks, support vector machines, classification trees and random forests (2011, methodology): “Traditional statistical classification methods (e.g., Fisher ’ s Linear Discriminant Analysis (LDA) and Logistic Regression (LR)) have been extensively used in medical classification problems for which the criterion variable is dichotomous [12-18].”
FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and Federated Image Classification (2022, methodology): “If the data is normally distributed, Naive Bayes is generally more effective than logistic regression [18, 19], and especially so in the small data setting [20].”