Convolutional Neural Networks for Text Categorization: Shallow Word-level vs. Deep Character-level

By Rie Johnson, Tong Zhang. 2016

50 citations recorded by the source corpus.

This paper reports the performances of shallow word-level convolutional neural networks (CNN), our earlier work (2015), on the eight datasets with relatively large training data that were used for testing the very deep character-level CNN in Conneau et al. (2016). Our findings are as follows. The shallow word-level CNNs achieve better error rates than the error rates reported in Conneau et al., though the results should be interpreted with some consideration due to the unique pre-processing of Conneau et al. The shallow word-level CNN uses more parameters and therefore requires more storage than the deep character-level CNN; however, the shallow word-level CNN computes much faster.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Deep Pyramid Convolutional Neural Networks for Text Categorization (2017, methodology): “Note that ShallowCNN enhanced with unsupervised embeddings (row 2) was originally proposed in (Johnson and Zhang, 2015b) as a semi-supervised extension of (Johnson and Zhang, 2015a), and then it was tested on the large datasets in (Johnson and Zhang, 2016).”
Adversarial Training Methods for Semi-Supervised Text Classification (2016, neutral): “These models now achieve state of the art Bag-of-words(Zhang et al., 2015) 3.57% Large-CNN(character-level) (Zhang et al., 2015) 1.73% SA-LSTM(word-level)(Dai & Le, 2015) 1.41% N-grams TFIDF (Zhang et al., 2015) 1.31% SA-LSTM(character-level)(Dai & Le, 2015) 1.19% Word CNN (Johnson & Zhang, 2016a) 0”
Hash Embeddings for Efficient Word Representations (2017, background): “This generally moves the task of ﬁnding meaningful representations of the text from the input embeddings into the model itself and increases the computational cost of running the models (Johnson and Zhang, 2016).”