Investigating Capsule Networks with Dynamic Routing for Text Classification

By Wei Zhao, Jianbo Ye, Min Yang, Zeyang Lei, Suofei Zhang, Zhou Zhao. Conference on Empirical Methods in Natural Language Processing, 2018

384 citations recorded by the source corpus.

In this study, we explore capsule networks with dynamic routing for text classification. We propose three strategies to stabilize the dynamic routing process to alleviate the disturbance of some noise capsules which may contain “background” information or have not been successfully trained. A series of experiments are conducted with capsule networks on six text classification benchmarks. Capsule networks achieve state of the art on 4 out of 6 datasets, which shows the effectiveness of capsule networks for text classification. We additionally show that capsule networks exhibit significant improvement when transfer single-label to multi-label text classification over strong baseline methods. To the best of our knowledge, this is the first work that capsule networks have been empirically investigated for text modeling.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/6a377115d0026bca3f67ed34ea03543880f6a2e3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

Transfer Capsule Network for Aspect Level Sentiment Classification (2019, methodology): “Recently, CapsNet has been applied to several NLP tasks like text classiﬁcation and relation extraction (Yang et al., 2018b; Gong et al., 2018; Xiao et al., 2018; Zhang et al., 2018; Wang et al., 2018b).”
Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications (2019, methodology): “, 2017) using holographic dual LSTM and CapsuleZhao (Zhao et al., 2018) using capsule networks.”
Fast dynamic routing based on weighted kernel density estimation (2018, positive): “Dynamic routing has been proven as an effective approach with higher generalization capacity and fewer parameters [6,7].”
Reconstructing Capsule Networks for Zero-shot Intent Classification (2019, background): “Recently, some studies have attempted to apply capsule networks in the domain of natural language processing (Yang et al., 2018; Geng et al., 2019; Xia et al., 2018) and obtained promising results.”
A Survey on Text Classification: From Traditional to Deep Learning (2020, methodology): “The SICK contains almost 10,000 English sentence pairs.”
A Survey on Text Classification: From Shallow to Deep Learning (2020, background): “5:19 Table 3. Summary statistics for the datasets. C: Number of target classes. L: Average sentence length. N: Dataset size. Datasets C L N Related Papers Sources Applications MR 2 20 10,662 [18] [7] [78] [6] [174] SA SST-1 5 18 11,855 [33] [18] [2] [3][50] [175] SA SST-2 2 19 9,613 [33] [18] [8] [4”
Induction Networks for Few-Shot Text Classification (2019, methodology): “Amazon Review Sentiment Classification (ARSC) Following Yu et al. (2018), we use the multiple tasks with the multi-domain sentiment classification (Blitzer et al.”
A Capsule Network for Recommendation and Explaining What You Like and Dislike (2019, methodology): “at low-level features can be selectively aggregated to form high-level features [22]. Recently, this notion has been applied to some NLP tasks, including relation extraction [36], text classification [35], zero-shot user intent detection [33] and multi-task learning [34]. In this work, we exploit th”
Multi-document Summarization via Deep Learning Techniques: A Survey (2020, methodology): “Capsule networks [111, 137] are employed to identify redundant information by transforming the spatial and orientational relationships for high-level representation.”
A CNN-Assisted Enhanced Audio Signal Processing for Speech Emotion Recognition (2019, background): “Recently researchers are trying to finding the robust and salient features for SER using artificial intelligence and deep learning approaches [3] to extracting hidden information, CNN features to trained different CNN models [4,5] to increasing the performance and decreasing the computational comple”
Deep Learning--based Text Classification (2020, methodology): “Their model’s architecture is similar to the ones in References [71, 72, 74].”
Joint Slot Filling and Intent Detection via Capsule Neural Networks (2018, background): “Recently, various capsule based text classiﬁcation models are proposed that aggregate word-level features for utterance-level classiﬁcation via dynamic routing-by-aggre ment (Gong et al., 2018; Zhao et al., 2018; Xia et al., 2018).”
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance (2019, methodology): “We study two alternatives for φ : (i) the concatenation of power means (R¨uckl´e et al., 2018) as a generalized pooling mechanism, and (ii) a routing mechanism for aggregation (Zhao et al., 2018, 2019).”
Automatic Text Evaluation through the Lens of Wasserstein Barycenters (2021, background): “…2018; Dong et al., 2019; Liu and Lapata, 2019; Chen and Bansal, 2018; See et al., 2017) and 14 abstractive systems (Zhong et al., 2020; Wang et al., 2020; Zhong et al., 2019; Liu and Lapata, 2019; Zhou et al., 2018; Narayan et al., 2018; Dong et al., 2019; Kedzie et al., 2018; Zhou et al., 2018).”
Explainable Sentiment Analysis with Applications in Medicine (2018, background): “However, for what concern Sentiment Analysis tasks, even if some efforts were made in this direction [38], [37], a more in-depth evaluation has still to be made.”