Comparing automated text classification methods

By Jochen Hartmann, Juliana Huppertz, Christina Schamp, Mark Heitmann. International Journal of Research in Marketing, 2019

347 citations recorded by the source corpus.

Abstract Online social media drive the growth of unstructured text data. Many marketing applications require structuring this data at scales non-accessible to human coding, e.g., to detect communication shifts in sentiment or other researcher-defined content categories. Several methods have been proposed to automatically classify unstructured text. This paper compares the performance of ten such approaches (five lexicon-based, five machine learning algorithms) across 41 social media datasets covering major social media platforms, various sample sizes, and languages. So far, marketing research relies predominantly on support vector machines (SVM) and Linguistic Inquiry and Word Count (LIWC). Across all tasks we study, either random forest (RF) or naive Bayes (NB) performs best in terms of correctly uncovering human intuition. In particular, RF exhibits consistently high performance for three-class sentiment, NB for small samples sizes. SVM never outperform the remaining methods. All lexicon-based approaches, LIWC in particular, perform poorly compared with machine learning. In some applications, accuracies only slightly exceed chance. Since additional considerations of text classification choice are also in favor of NB and RF, our results suggest that marketing research can benefit from considering these alternatives.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/51d1f2d498046021d9387da88c63c8535c973a02

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Roland T. Rust
2. Ming-Hui Huang
3. Daniel Belanche
4. Luis Vicente Casaló
5. Carlos Flavián

Citations in the corpus

• A strategic framework for artificial intelligence in marketing (2020, background): “At the feeling level, more real-time and accurate emotion sensing from posted messages can better engage customers and provide a better interaction experience (Hartmann et al. 2019; Lee et al. 2018).”
• Machine learning and AI in marketing – Connecting computing power to human insights (2020, neutral): “In accordance with their traditional focus, machine learning methods have been used for prediction in marketing research (Cui & Curry, 2005; Dzyabura et al., 2019; Hartmann, Huppertz, et al., 2019; Huang & Luo, 2016).”