Character-level Convolutional Networks for Text Classification
By Junbo Jake Zhao, Yann LeCun. 2015
7,191 citations recorded by the source corpus.
This article offers an empirical exploration on the use of character-level convolutional networks (ConvNets) for text classification. We constructed several large-scale datasets to show that character-level convolutional networks could achieve state-of-the-art or competitive results. Comparisons are offered against traditional models such as bag of words, n-grams and their TFIDF variants, and deep learning models such as word-based ConvNets and recurrent neural networks.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Graham Neubig
- Yejin Choi
- Noah A. Smith
- Yoshua Bengio
In Natural Language Processing
- Mohit Bansal
- Luke S. Zettlemoyer
- Graham Neubig
- Jianfeng Gao
- Yejin Choi
- Noah A. Smith
- Kyunghyun Cho
- Percy Liang
Citations in the corpus
- QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension (2018, background): “For example, Zhang et al. (2015) proposed to enhance the dataset by replacing the words with their synonyms and showed its effectiveness in text classification.”
- Universal Language Model Fine-tuning for Text Classification (2018, methodology): “Topic classification For topic classification, we evaluate on the large-scale AG news and DBpedia ontology datasets created by Zhang et al. (2015).”
- Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning (2017, background): “These encoders are typically implemented as convolutional (Kim, 2014), recursive (Socher et al., 2013), or recurrent neural networks (Mikolov et al., 2010) operating over a sentence’s words or characters (Zhang et al., 2015; Kim et al., 2016).”
- Semi-supervised Sequence Learning (2015, methodology): “For this experiment, we follow the same procedure suggested in Zhang and LeCun [38].”
- XLNet: Generalized Autoregressive Pretraining for Language Understanding (2019, methodology): “Following previous work on text classification [40, 20], we evaluate XLNet on the following benchmarks: IMDB, Yelp-2, Yelp-5, DBpedia, AG, Amazon-2, and Amazon-5.”
- Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference (2020, methodology): “We use two vertical bars (‖) to mark boundaries between text segments.1 Yelp For the Yelp Reviews Full Star dataset (Zhang et al., 2015), the task is to estimate the rating that a customer gave to a restaurant on a 1- 1The way different segments are handled depends on the language model being used;…”
- Unsupervised Data Augmentation for Consistency Training (2019, neutral): “These methods employ mixup (Zhang et al., 2017) on top of simple augmentations such as flipping and cropping; instead, UDA emphasizes on the use of state-of-the-art data augmentations, leading to significantly better results on CIFAR-10 and SVHN.”
- Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning (2018, neutral): “DBpedia is a large corpus of sentences from Wikipedia labeled by category and used by Zhang et al. (2015).”
- Fine-tuned Language Models for Text Classification (2018, methodology): “…and TREC-6 datasets, we compare against CoVe (McCann et al., 2017), a state- of-the-art transfer learning method for NLP.5 For the AG-News, Yelp, and DBpedia datasets compiled by Zhang et al. (2015), we compare against the state-of-the-art text categorization method by Johnson and Zhang (2017).”
- Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function (2019, methodology): “For ACL-IMDB, Elec, RCV1, IMDB, and Arxiv datasets, we use training and test set as unlabeled data, while for AG-News and DBpedia datasets as their test sets are small, we use only the training set as unlabeled data.”
- Deep Pyramid Convolutional Neural Networks for Text Categorization (2017, methodology): “Row 7 shows the performance of deep word-level CNN from (Zhang et al., 2015), which was designed to match their character-level models in complexity.”
- Adversarial Training Methods for Semi-Supervised Text Classification (2016, neutral): “These models now achieve state of the art Bag-of-words(Zhang et al., 2015) 3.57% Large-CNN(character-level) (Zhang et al., 2015) 1.73% SA-LSTM(word-level)(Dai & Le, 2015) 1.41% N-grams TFIDF (Zhang et al., 2015) 1.31% SA-LSTM(character-level)(Dai & Le, 2015) 1.19% Word CNN (Johnson & Zhang, 2016a) 0”
- Finetuned Language Models Are Zero-Shot Learners (2021, neutral): “AG News (Zhang et al., 2015) 9.”
- Self-Refine: Iterative Refinement with Self-Feedback (2023, methodology): “Dataset: (Zhang et al., 2015) 1000 review passages x: The food was fantastic.”
- Enriching Word Vectors with Subword Information (2016, methodology): “These different approaches rely on a morphological decomposition of words, while ours does not.”
- Fine-Grained Image Classification via Combining Vision and Language (2017, methodology): “Besides LSTMs, character-based convolutional networks [28] have been used for language modeling.”
- On Sampling Strategies for Neural Network-based Collaborative Filtering (2017, background): “Deep Neural Networks (DNNs) have been showing extraordinary abilities to extract high-level features from raw data, such as video, audio, and text [8, 15, 34].”