An Improved k-Nearest Neighbor Algorithm for Text Categorization

By Baoli Li, Shiwen Yu, Qin Lu. 2003

233 citations recorded by the source corpus.

k is the most important parameter in a text categorization system based on k-Nearest Neighbor algorithm (kNN).In the classification process, k nearest documents to the test one in the training set are determined firstly. Then, the predication can be made according to the category distribution among these k nearest neighbors. Generally speaking, the class distribution in the training set is uneven. Some classes may have more samples than others. Therefore, the system performance is very sensitive to the choice of the parameter k. And it is very likely that a fixed k value will result in a bias on large categories. To deal with these problems, we propose an improved kNN algorithm, which uses different numbers of nearest neighbors for different categories, rather than a fixed number across all categories. More samples (nearest neighbors) will be used for deciding whether a test document should be classified to a category, which has more samples in the training set. Preliminary experiments on Chinese text categorization show that our method is less sensitive to the parameter k than the traditional one, and it can properly classify documents belonging to smaller classes with a large k. The method is promising for some cases, where estimating the parameter k via cross-validation is not allowed.

Subfields: Artificial Intelligence, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

A Comparative Analysis of Logistic Regression, Random Forest and KNN Models for the Text Classification (2020, background): “[21] also improved the K-nearest neighbour for the categorization of text.”
Efficient kNN Classification With Different Numbers of Nearest Neighbors (2018, background): “[32] proposed to use different numbers of nearest neighbors”
Survey on supervised machine learning techniques for automatic text classification (2019, methodology): “Instances of some methods contain a variant of feature selection procedure that they decrease the DR through selecting a subgroup of the original keywords (Li et al. 2003).”
Text mining for market prediction: A systematic review (2014, methodology): “Another less frequently used but interesting technique is the so called Latent Dirichlet Allocation (LDA) technique used by Jin et al. (2013) as well as Mahajan et al. (2008) in Table 3 to categorize words into concepts and use the representative concepts as the selected features.”
A survey on instance selection for active learning (2012, methodology): “Instance correlation Exploiting feature correlation [EFC] Cosine similarity [EFCCS] [29,38,47] [49]”