Tackling the Poor Assumptions of Naive Bayes Text Classifiers

By Jason D. M. Rennie, Lawrence Kai Shih, Jaime Teevan, David R Karger. International Conference on Machine Learning, 2003

1,297 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/b7fab2f72ebbf4e98def1daf8c29ffcfe91183bf

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Yoshua Bengio
2. Percy Liang
3. Mohit Bansal
4. Quoc V. Le
5. Sergey Levine
6. Zhe Gan
7. Dacheng Tao
8. Kyunghyun Cho

Citations in the corpus

• Multinomial Naive Bayes for Text Categorization Revisited (2004, methodology): “However, recently a new algorithm has been proposed, called “transformed weight-normalized complement naive Bayes” (TWCNB), that is easy to implement, has good running time and is claimed to be nearly as accurate as support vector machines [1].”
• Less is More Probabilistic Models for Retrieving Fewer Relevant Documents (2006, background): “To model the fact that we do not know exactly what terms appear in relevant and irrelevant distributions, we specify a prior probability distribution over the parameters (a distribution over possible document distributions).”
• Very sparse random projections (2006, methodology): “See similar comments in [37] for the work on Naive Bayes text classifier.”
• Multi-label classification with a reject option (2013, methodology): “data sets related to text categorisation we also used the ad hoc version of the Naive Bayes (NB) classifier of [26].”
• Reducing the effort of bug report triage: Recommenders for development-oriented decisions (2011, methodology): “We then normalize the frequencies based on document (i.e., combined title and description) length, intra-document frequency and inter-document frequency [Rennie et al. 2003].”
• Impact of license choice on Open Source Software development activity (2009, methodology): “Unlike the traditional naive Bayes text classifier, we as in Rennie, Teevan, and Karger (2003) do not consider prior probability while predicting the category for a testing document.As SVM only classifies documents into two classes (binary classifier), we use the one-against-all (Yi & Zheng, 2005)”
• Scruples: A Corpus of Community Ethical Judgments on 32, 000 Real-Life Anecdotes (2020, methodology): “(Rennie et al. 2003) apply Bernoulli, Multinomial, and Complement Naive Bayes, while Logistic and Forest apply logistic regression and random forests (Breiman 2001).”
• Comparison of Naive Bayes, Random Forest, Decision Tree, Support Vector Machines, and Logistic Regression Classifiers for Text Reviews Classification (2017, methodology): “To improve the classification, it was decided to split the data to equally distributed sets per each class and using the method for measuring the skewness of data (Rennie, 2003), so that each class would collect an equal number of customer product-review records.”
• A dirichlet multinomial mixture model-based approach for short text clustering (2014, background): “However, this is a good property in the text clustering problem, because words in a document tend to appear in bursts: if a word appears once, it is more likely to appear again [14],[23].”
• Learning Syntactic Patterns for Automatic Hypernym Discovery (2004, methodology): “We use the WordNet-labeled Known Hypernym / Known Non-Hypernym training set dened in Section 3. We train a number of classiers on this data set, including multinomial Naive Bayes, complement Naive Bayes [ 16 ], and logistic regression.”
• Naive Bayes: applications, variations and vulnerabilities: a review of literature with code snippets for implementation (2020, background): “Rennie et al. (2003) see the main assumption of the NB is very straight forward and simple which has helped it to become an easy to use algorithm.”
• Automating Large-Scale Data Quality Verification (2018, methodology): “Afterwards, any classification algorithm from SparkML [27] can be used to learn a prediction model; in our experiments we used Sparks Naive Bayes [36] implementation as it offers a scalable lower bound on prediction accuracy, does not require hyperparameter optimization, and is simple to train incre”
• Baselines and Bigrams: Simple, Good Sentiment and Topic Classification (2012, background): “Compared to the excellent performance of MNB on snippet datasets, the many poor assumptions of MNB pointed out in (Rennie et al., 2003) become more crippling for these longer documents.”
• Current and future applications of statistical machine learning algorithms for agricultural machine vision systems (2019, background): “Therefore, adding more predictor variable might not increase but decrease the overall classification accuracy because of their correlation (Rennie et al., 2003) (Table 1).”
• Tweets and Trades: The Information Content of Stock Microblogs (2010, unknown): “Finally, following established preprocessing procedures (see Rennie et al. (2003)), word counts are transformed to a power-law distributions that comes closer to empirical text distributions than most training sets (term frequency [TF] transformation) and words occurring in many messages are…”
• Feature Weighting for Improved Classifier Robustness (2009, background): “The combination of term weighting and document length normalization can make a prominent difference for some classifiers, such as Naive Bayes [14, 13].”