A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification

By Ye Zhang, Byron C. Wallace. 2015

1,309 citations recorded by the source corpus.

Convolutional Neural Networks (CNNs) have recently achieved remarkably strong performance on the practically important task of sentence classification (Kim, 2014; Kalchbrenner et al., 2014; Johnson and Zhang, 2014; Zhang et al., 2016). However, these models require practitioners to specify an exact model architecture and set accompanying hyperparameters, including the filter region size, regularization parameters, and so on. It is currently unknown how sensitive model performance is to changes in these configurations for the task of sentence classification. We thus conduct a sensitivity analysis of one-layer CNNs to explore the effect of architecture components on model performance; our aim is to distinguish between important and comparatively inconsequential design decisions for sentence classification. We focus on one-layer CNNs (to the exclusion of more complex models) due to their comparative simplicity and strong empirical performance, which makes it a modern standard baseline method akin to Support Vector Machine (SVMs) and logistic regression. We derive practical advice from our extensive empirical results for those interested in getting the most out of CNNs for sentence classification in real world settings.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, background): “Sensitivity refers to how an ANN output is influenced by its input and/or weight perturbations [128].”
Recent Trends in Deep Learning Based Natural Language Processing (2017, methodology): “6: CNN modeling on text (Figure source: Zhang and Wallace [46])”
Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling (2016, methodology): “(Zhang and Wallace, 2015) conducted a sensitivity analysis of one-layer CNN to explore the effect of architecture components on model performance.”
Rationale-Augmented Convolutional Neural Networks for Text Classification (2016, methodology): “5, and we used 3 different filter heights: 3, 4 and 5, following (Zhang and Wallace, 2015).”
Multichannel Convolutional Neural Network for Biological Relation Extraction (2016, methodology): “According to the suggestion that the best window size combination is usually close to each other by Zhang and Wallace [35], we set the windows size ℎ as [5, 6, 7, 8] in DDI task and [3, 4, 5, 6] in PPI task.”
Deep Learning--based Text Classification (2020, background): “Zhang et al. [60] examine the impact of different word embedding methods and pooling mechanisms, and find that using non-static word2vec and GloVe outperforms one-hot vectors, and that max-pooling consistently outperforms other pooling methods.”
Do Convolutional Networks need to be Deep for Text Classification ? (2017, methodology): “Figure 1: Shallow-and-wide CNN, from (Zhang and Wallace 2015): 3 convolutional layers with respective kernel window sizes 3,4,5 are used.”
Recent advances in deep learning based dialogue systems: a systematic survey (2021, methodology): “Figure 2: A CNN architecture for text classification (Zhang and Wallace, 2017)”
A review on deep learning for recommender systems: challenges and remedies (2018, background): “These features can be used within the fully connected layers for classification (Zhang and Wallace 2015).”
Analysis of year-over-year changes in Risk Factors Disclosure in 10-K filings (2018, background): “Also, CNNs maintain the semantics of every word since the entire sentence is encoded as a vector and is fed as an input to the network [16, 22].”
MGNC-CNN: A Simple Approach to Exploiting Multiple Word Embeddings for Sentence Classification (2016, background): “This replication is important because training is stochastic and thus introduces variance in performance (Zhang and Wallace, 2015).”
Sentiment analysis using deep learning architectures: a review (2019, background): “Motivated by these results, Zhang and Wallace (2015) discussed an architecture for sentence classification using one-layer CNN.”
SGM: Sequence Generation Model for Multi-label Classification (2018, methodology): “In addition, training strategies mentioned in Zhang and Wallace (2015) are used to tune hyper-parameters for the baselines CNN and CNN-RNN.”
Neural information retrieval: at the end of the early years (2017, methodology): “Zhang and Wallace (2015) perform an empirical analysis in the context of using CNNs for short text classification.”
Deeper Attention to Abusive User Content Moderation (2017, result): “We describe the CNN only briefly, because it is very similar to that of of Kim (2014); see also Goldberg (2016) for an introduction to CNNs, and Zhang and Wallace (2015).”
Show Your Work: Improved Reporting of Experimental Results (2019, result): “1) confirm previous findings (Zhang and Wallace, 2015): under a budget of fewer than 10 hyperparameter”