Social Media Text Classification under Negative Covariate Shift
40 citations recorded by the source corpus.
In a typical social media content analysis task, the user is interested in analyzing posts of a particular topic. Identifying such posts is often formulated as a classification problem. However, this problem is challenging. One key issue is covariate shift. That is, the training data is not fully representative of the test data. We observed that the covariate shift mainly occurs in the negative data because topics discussed in social media are highly diverse and numerous, but the user-labeled negative training data may cover only a small number of topics. This paper proposes a novel technique to solve the problem. The key novelty of the technique is the transformation of document representation from the traditional ngram feature space to a center-based similarity (CBS) space. In the CBS space, the covariate shift problem is significantly mitigated, which enables us to build much better classifiers. Experiment results show that the proposed approach markedly improves classification.
Subfields: Artificial Intelligence, Information Systems
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Information Systems
Citations in the corpus
- DOC: Deep Open Classification of Text Documents (2017, methodology): “Fei and Liu (2016) proposed a Center-Based Similarity (CBS) space learning method (Fei and Liu, 2015).”
- Learning Cumulatively to Become More Knowledgeable (2016, methodology): “The similarity measures are the five ones described in [13], which produce 10 CBS features to represent a document in the CBS space.”
- Breaking the Closed World Assumption in Text Classification (2016, methodology): “We will also discuss its underlying assumptions, which was not addressed in the original paper of Fei and Liu (2015).”