Densely Connected CNN with Multi-scale Feature Attention for Text Classification
By Shiyao Wang, Minlie Huang, Zhidong Deng. 2018
179 citations recorded by the source corpus.
Text classification is a fundamental problem in natural language processing. As a popular deep learning model, convolutional neural network (CNN) has demonstrated great success in this task. However, most existing CNN models apply convolution filters of fixed window size, thereby unable to learn variable n-gram features flexibly. In this paper, we present a densely connected CNN with multi-scale feature attention for text classification. The dense connections build short-cut paths between upstream and downstream convolutional blocks, which enable the model to compose features of larger scale from those of smaller scale, and thus produce variable n-gram features. Furthermore, a multi-scale feature attention is developed to adaptively select multi-scale features for classification. Extensive experiments demonstrate that our model obtains competitive performance against state-of-the-art baselines on five benchmark datasets. Attention visualization further reveals the model's ability to select proper n-gram features for text classification.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Erik Cambria
- Shervin Minaee
- Jianfeng Gao
- Soujanya Poria
- Ekaterina Artemova
- Nikita Klyuchnikov
- Maxim V. Fedorov
- Ilya Trofimov
Citations in the corpus
- NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processing (2020, methodology): “Residual connections [16] and dense connections [19] were adopted from computer vision architectures and impacted on machine translation [7], text classification [43], and machine reading comprehension [40].”
- Deep Learning--based Text Classification (2020, background): “[90] propose a densely connected CNN with multi-scale feature attention to produce variable n-gram features.”
- Paying Attention to Attention: Highlighting Influential Samples in Sequential Analysis (2018, methodology): “The classifier (H AN ) used for escalation prediction is outlined in (Yang et al., 2016).”
- Bi-LSTM Model to Increase Accuracy in Text Classification: Combining Word2vec CNN and Attention Mechanism (2020, methodology): “…model has two major problems. in one fixed-size vector, there can be instances of Since all the information cannot be compressed in one fixed-size vector, there can be instances of information loss, and a vanishing gradient problem [18]. information loss, and a vanishing gradient problem [18].”