Variational Autoencoder for Semi-Supervised Text Classification

By Weidi Xu, Haoze Sun, Chao Deng, Ying Tan. 2017

236 citations recorded by the source corpus.

Although semi-supervised variational autoencoder (SemiVAE) works in image classification task, it fails in text classification task if using vanilla LSTM as its decoder. From a perspective of reinforcement learning, it is verified that the decoder's capability to distinguish between different categorical labels is essential. Therefore, Semi-supervised Sequential Variational Autoencoder (SSVAE) is proposed, which increases the capability by feeding label into its decoder RNN at each time-step. Two specific decoder structures are investigated and both of them are verified to be effective. Besides, in order to reduce the computational complexity in training, a novel optimization method is proposed, which estimates the gradient of the unlabeled objective function by sampling, along with two variance reduction techniques. Experimental results on Large Movie Review Dataset (IMDB) and AG's News corpus show that the proposed approach significantly improves the classification accuracy compared with pure-supervised classifiers, and achieves competitive performance against previous advanced methods. State-of-the-art results can be obtained by integrating other pretraining-based methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines (2018, neutral): “Text classification following a semi-supervised approach by means of AEs is introduced in [117].”
Automatically Learning Data Augmentation Policies for Dialogue Tasks (2019, neutral): “…Lemley et al., 2017) and NLP (F¨urstenau and Lapata, 2009; Sen-nrich et al., 2016; Wang and Yang, 2015; Zhang et al., 2015; Jia and Liang, 2016; Kim and Rush, 2016; Hu et al., 2017; Xu et al., 2017; Xia et al., 2017; Silfverberg et al., 2017; Kaﬂe et al., 2017; Hou et al., 2018; Wang et al., 2018).”
Lifelong Generative Modeling (2017, methodology): “M L ] 2 7 M ay of leveraging our model for tasks such as unsupervised learning or even semi-supervised learning Xu et al. (2017).”
Structured Disentangled Representations (2018, positive): “The latent code z can also serve as a compressed representation for downstream tasks such as text classification [Xu et al., 2017], Bayesian optimization [Gómez-Bombarelli et al.”
Deconvolutional Latent-Variable Model for Text Sequence Matching (2017, methodology): “The proposed framework is closely related to recent research on incorporating NVI into text modeling (Bowman et al. 2016b; Miao, Yu, and Blunsom 2016; Xu et al. 2017; Zhang et al. 2016; Serban et al. 2017).”
Towards Explainable NLP: A Generative Explanation Framework for Text Classification (2018, methodology): “Unfortunately, existing text classification models typically utilize the original review texts to predict the overall results (Tang et al., 2015b; Asghar, 2016; Xu et al., 2017) directly, but none of them provides fine-grained explanations about why the decisions are made.”
Learning With Single-Teacher Multi-Student (2018, background): “Other MC applications refer to image classification (Guo 2017), text categorization (Xu et al. 2017), information retrieval (Yang et al. 2017), multi-view classification (Xu, Tao, and Xu 2013) et.al.”
A Survey on Deep Semi-Supervised Learning (2021, background): “Text classification AG News [95], BPpedia [96], Yahoo! Answers [97], IMDB [98], Yelp review [95], Snippets [99], Ohsumed [100], TagMyNews [101], MR [102], Elec [103], Rotten Tomatoes [102], RCV1 [104] [105], [106], [107], [108], [109], [110]”
Semi-Supervised Dialogue Policy Learning via Stochastic Reward Estimation (2020, methodology): “, sentiment analysis (Xu et al., 2017; Li et al., 2019a), text matching (Shen et al.”
Hierarchical Disentangled Representations (2018, methodology): “The smooth low-dimensional z can be used as a compressed representation for downstream tasks such as text classification [Xu et al., 2017], Bayesian optimization [Gómez-Bombarelli et al.”
Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing (2019, background): “…NLP tasks, including language modeling (Bowman et al., 2015; Miao et al., 2016), dialog response generation (Zhao et al., 2017; Wen et al., 2017), semi-supervised text classiﬁ-cation (Xu et al., 2017), controllable text generation (Hu et al., 2017), and text compression (Miao and Blunsom, 2016).”
Learning to Annotate: Modularizing Data Augmentation for Text Classifiers with Natural Language Explanations (2019, methodology): “Meanwhile, if we regard those data that are exactly matched by rules as labeled data and the remaining as unlabeled data, we can apply many semi-supervised models such as self learning (Rosenberg et al., 2005), meanteacher (Tarvainen and Valpola, 2017), and semisupervised VAE (Xu et al., 2017).”
Weakly-Supervised Neural Text Classification (2018, methodology): “pervised text classification. However, training neural models for text classification under weak supervision remains an open research problem. Several semi-supervised neural models have been proposed [18, 31], but they still require hundreds or even thousands of labeled training examples, which are ”
Spherical Latent Spaces for Stable Variational Autoencoders (2018, background): “VAEs can be also be applied for semi-supervised classification (Xu et al., 2017).”
Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations (2018, background): “, 2017), or neural decoders of autoencoders (Bergmanis et al., 2017; Xu et al., 2017; Hu et al., 2017) or encoder-decoder models (Kim and Rush, 2016; Sennrich et al.”
An Introduction to Variational Autoencoders (2019, background): “The application to semi-supervised learning has been studied in (Kingma et al., 2014; Pu et al., 2016; Xu et al., 2017) among other work.”