Deconvolutional Latent-Variable Model for Text Sequence Matching

By Dinghan Shen, Yizhe Zhang, Ricardo Henao, Qinliang Su, Lawrence Carin. 2017

69 citations recorded by the source corpus.

A latent-variable model is introduced for text matching, inferring sentence representations by jointly optimizing generative and discriminative objectives. To alleviate typical optimization challenges in latent-variable models for text, we employ deconvolutional networks as the sequence decoder (generator), providing learned latent codes with more semantic information and better generalization. Our model, trained in an unsupervised manner, yields stronger empirical predictive performance than a decoder based on Long Short-Term Memory (LSTM), with less parameters and considerably faster training. Further, we apply it to text sequence-matching problems. The proposed model significantly outperforms several strong sentence-encoding baselines, especially in the semi-supervised setting.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning (2018, methodology): “We also outperform the Deconv LVM model proposed by Shen et al. (2017) in this low-resource setting.”
Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization (2018, methodology): “For all the experiments, we employed a 3-layer convolutional encoder and an LSTM decoder as in [45].”
Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms (2018, background): “The Convolutional Neural Network (CNN) architecture (Kim, 2014; Collobert et al., 2011; Gan et al., 2017; Zhang et al., 2017b; Shen et al., 2018) is another strategy extensively employed as the compositional function to encode text sequences.”
Joint Embedding of Words and Labels for Text Classification (2018, neutral): “…have been employed to learn text representations, including convolutional neural networks (CNNs) (Kalchbrenner et al., 2014; Zhang et al., 2017b; Shen et al., 2017) and recurrent neural networks (RNNs) based on long short-term memory (LSTM) (Hochreiter and Schmidhuber, 1997; Wang et al., 2018).”
How to Fine-Tune BERT for Text Classification? (2019, neutral): “…including convolution models (Kalchbrenner et al., 2014; Zhang et al., 2015; Conneau et al., 2016; Johnson and Zhang, 2017; Zhang et al., 2017; Shen et al., 2018), recurrent models (Liu et al., 2016; Yogatama et al., 2017; Seo et al., 2017), and attention mechanisms (Yang et al., 2016; Lin et…”
Adversarial Symmetric Variational Autoencoder (2017, background): “Variational autoencoders (VAEs) are designed to learn both an encoder and decoder, leading to excellent data reconstruction and the ability to quantify a bound on the log-likelihood fit of the model to data [12, 13, 14, 15, 16, 17, 18, 19].”
Dual Latent Variable Model for Low-Resource Natural Language Generation in Dialogue Systems (2018, methodology): “To solve the exposure-bias problem (Bengio et al., 2015) Zhang et al. (2017); Shen et al. (2017) proposed a seq2seq purely convolutional and deconvolutional autoencoder, Yang et al. (2017) proposed to use a dilated CNN decoder in a latentvariable model, or Semeniuta et al. (2017) proposed a hybrid…”
Improved Semantic-Aware Network Embedding with Fine-Grained Word Alignment (2018, background): “In future work, we aim to leverage abundant unlabeled text data to abstract more informative sentence representations (Dai and Le, 2015; Zhang et al., 2017; Shen et al., 2017; Tang and de Sa, 2018) .”
Learning Context-Sensitive Convolutional Filters for Text Processing (2017, background): “, 2018), text sequence matching (Hu et al., 2014; Yin et al., 2016; Shen et al., 2017), generic text representations (Gan et al.”
On the Use of Word Embeddings Alone to Represent Natural Language Sequences (2018, background): “Examples of this are sentence/document classiﬁcation (Le & Mikolov, 2014; Zhang et al., 2015), text-sequence matching (Hu et al., 2014; Shen et al., 2017), machine translation (Bahdanau et al., 2014), etc.”
Zero-Shot Learning via Class-Conditioned Deep Generative Models (2017, background): “Deep encoder-decoder based models have recently gained much attention for a variety of problems, ranging from image generation (Rezende et al. 2016) and text matching (Shen et al. 2017).”
NASH: Toward End-to-End Neural Architecture for Generative Semantic Hashing (2018, methodology): “Inspired by the recent success of variational autoencoders for various NLP problems (Miao et al., 2016; Bowman et al., 2015; Yang et al., 2017; Miao et al., 2017; Shen et al., 2017b; Wang et al., 2018), we approach the training of discrete (binary) latent variables from a generative perspec-”