Deep Learning--based Text Classification

By Shervin Minaee, Erik Cambria, Jianfeng Gao. 2020

1,342 citations recorded by the source corpus.

Deep learning--based models have surpassed classical machine learning--based approaches in various text classification tasks, including sentiment analysis, news categorization, question answering, and natural language inference. In this article, we provide a comprehensive review of more than 150 deep learning--based models for text classification developed in recent years, and we discuss their technical contributions, similarities, and strengths. We also provide a summary of more than 40 popular datasets widely used for text classification. Finally, we provide a quantitative analysis of the performance of different deep learning models on popular benchmarks, and we discuss future research directions.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Adversarial Training for Large Neural Language Models (2020, background): “In NLP, applying adversarial training is not straightforward, since the input are discrete elements (token or subword sequences), but there have been some recent successes (Zhu et al., 2019; Jiang et al., 2019; Cheng et al., 2019; Wang et al., 2019; Minaee et al., 2020b).”
DeBERTa: Decoding-enhanced BERT with Disentangled Attention (2020, methodology): “These PLMs have been fine-tuned using task-specific labels and created new state-of-the-art in many downstream natural language processing (NLP) tasks [13, 14, 15, 16, 17, 18].”
A Survey on Data Augmentation for Text Classification (2021, neutral): “In this survey, text classification is considered a problem of the field of NLP, where units such as sentences, paragraphs, or documents are categorized into class labels [17].”
A Survey on Text Classification: From Traditional to Deep Learning (2020, background): “[13] reviewed recent deep learning based text classificationmethods, benchmark datasets, and evaluation metrics.”
An Empirical Survey of Data Augmentation for Limited Data Learning in NLP (2021, background): “Deep learning methods have achieved strong performance on a wide range of supervised learning tasks (Sutskever et al., 2014; Deng et al., 2013; Minaee et al., 2021).”
A Survey on Text Classification: From Shallow to Deep Learning (2020, background): “Minaee et al. [14] reviewed recent deep learning based text classification methods, benchmark datasets, and evaluation metrics.”
Exploring AI Ethics of ChatGPT: A Diagnostic Analysis (2023, background): “models to make informed decisions based on natural language input, as demonstrated in studies on sentiment analysis [36], text classification [37], and question answering [38].”
Preserving integrity in online social networks (2020, background): “Building on the long history of text classification research in the literature, many studies have been proposed to address different aspects in understanding text [11, 53, 65, 78].”