Understanding Convolutional Neural Networks for Text Classification

By Alon Jacovi, Oren Sar Shalom, Yoav Goldberg. 2018

243 citations recorded by the source corpus.

We present an analysis into the inner workings of Convolutional Neural Networks (CNNs) for processing text. CNNs used for computer vision can be interpreted by projecting filters into image space, but for discrete sequence inputs CNNs remain a mystery. We aim to understand the method by which the networks process and classify text. We examine common hypotheses to this problem: that filters, accompanied by global max-pooling, serve as ngram detectors. We show that filters may capture several different semantic classes of ngrams by using different activation patterns, and that global max-pooling induces behavior which separates important ngrams from the rest. Finally, we show practical use cases derived from our findings in the form of model interpretability (explaining a trained model by deriving a concrete identity for each filter, bridging the gap between visualization tools in vision tasks and NLP) and prediction interpretability (explaining predictions).

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

SELFEXPLAIN: A Self-Explaining Architecture for Neural Text Classifiers (2021, background): “Sufficiency aims to evaluate whether model explanations alone are highly indicative of the predicted label (Jacovi et al., 2018; Yu et al., 2019).”
Transformer Feed-Forward Layers Are Key-Value Memories (2020, background): “Jacovi et al. (2018) analyzed CNN architectures in text classification and showed that they extract key n-grams from the inputs.”
Analysis Methods in Neural Language Processing: A Survey (2018, background): “Generally less work has analyzed convolutional neural networks in NLP, but see Jacovi et al. (2018) for a recent exception.”
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI (2019, methodology): “Similarly, [40] computes the scores reflecting the n-grams activating convolution filters in NLP (Natural Language Processing).”
Do Not Trust Additive Explanations (2019, methodology): “, 2017), text data (Marco Tulio Ribeiro and Sameer Singh and Carlos Guestrin, 2018; Jacovi et al., 2018; Alvarez-Melis and Jaakkola, 2017), and tabular data (Molnar, 2019; Biecek, 2018).”
Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness? (2020, background): “Otherwise, the “least relevant” parts of the input may be erased, in expectation that the model’s decision will not change (Jacovi et al., 2018).”
Human-in-the-loop Debugging Deep Text Classifiers (2020, methodology): “This is similar to how previous works analyze CNN features (Jacovi et al., 2018; Lertvittayakumjorn and Toni, 2019), and it is equivalent to back-propagating the feature values to the input using LRP and cropping the consecutive input words with non-zero LRP scores to show in the word clouds.”
Trustworthy Artificial Intelligence: A Review (2022, background): “In-the-loop 3 Interpretable Model Global [40, 87, 120] Feature Importance Local [161] Global, Local [9, 66, 77, 93, 127, 227]”
KERMIT: Complementing Transformer Architectures with Encoders of Explicit Syntactic Interpretations (2020, methodology): “diction interpretability (Jacovi et al., 2018) in the context of textual classification.”
Learning and Understanding Different Categories of Sexism Using Convolutional Neural Network’s Filters (2019, methodology): “After running the model and fine-tuning, we proceed to interpret the model in accordance with methods introduced by Jacovi et al. (2018).”
Getting to Production with Few-shot Natural Language Generation Models (2021, methodology): “We train two models, a production-grade convolutional (DocNN) model (Jacovi et al., 2018) with median latency of 8 ms and a high-capacity pre-trained RoBERTa-Base model (Liu et al., 2019) with latency 100 ms.”
Lucid: A Practical, Lightweight Deep Learning Solution for DDoS Attack Detection (2020, methodology): “This approach is inspired by a similar study [65] to interpret CNNs in the rather different domain of natural language pro-”
A survey on fake news and rumour detection techniques (2019, background): “However, CNNs are gaining popularity in the NLP community as well [49].”
Human-grounded Evaluations of Explanation Methods for Text Classification (2019, background): “We found that Jacovi et al. (2018) proposed one: listing only n-grams corresponding to feature values in v (see Figure 1) that pass thresholds for their ﬁlters.”