LSTM-based Deep Learning Models for non-factoid answer selection
By Ming Tan, Bing Xiang, Bowen Zhou. 2015
439 citations recorded by the source corpus.
In this paper, we apply a general deep learning (DL) framework for the answer selection task, which does not depend on manually defined features or linguistic tools. The basic framework is to build the embeddings of questions and answers based on bidirectional long short-term memory (biLSTM) models, and measure their closeness by cosine similarity. We further extend this basic model in two directions. One direction is to define a more composite representation for questions and answers by combining convolutional neural network with the basic framework. The other direction is to utilize a simple but efficient attention mechanism in order to generate the answer representation according to the question context. Several variations of models are provided. The models are examined by two datasets, including TREC-QA and InsuranceQA. Experimental results demonstrate that the proposed models substantially outperform several strong baselines.
Subfields: Artificial Intelligence, Question Answering & Reading Comprehension
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Luke S. Zettlemoyer
- Percy Liang
- Caiming Xiong
- Richard Socher
- Noah A. Smith
- Christopher D. Manning
In Question Answering & Reading Comprehension
Citations in the corpus
- Sentence-State LSTM for Text Representation (2018, methodology): “, 2015), syntactic parsing (Dozat and Manning, 2017) and question answering (Tan et al., 2015).”
- ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs (2015, methodology): “Yang et al. (2015) extend this method and get state-of-the-art performance on the the WikiQA dataset (Section 5.”
- Attentive Pooling Networks (2016, methodology): “Different neural network architectures have been recently proposed to perform matching of semantically related text segments (Yu et al., 2014; Hu et al., 2014; dos Santos et al., 2015; Wang & Nyberg, 2015; Severyn & Moschitti, 2015; Tan et al., 2015).”
- Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention (2016, methodology): “(Tan et al., 2015) A recent work by (Rockt¨aschel et al., 2015) improved the performance by applying a neural attention model that didn’t yield sentence embeddings.”
- Bilateral Multi-Perspective Matching for Natural Language Sentences (2017, methodology): “…Siamese architecture [Bromley et al. , 1993], where sentences are encoded into sentence vectors based on some neural network encoders, and then the relationship between two sentences was decided solely based on the two sentence vectors [Bowman et al. , 2015; Yang et al. , 2015; Tan et al. , 2015].”
- Semi-supervised Question Retrieval with Gated Convolutions (2015, background): “Recent work on answer selection on community QA forums, similar to our task of question retrieval, has also involved the use of neural network architectures (Severyn and Moschitti, 2015; Wang and Nyberg, 2015; Shen et al., 2015; Feng et al., 2015; Tan et al., 2015).”
- Knowledge Distillation in Document Retrieval (2019, background): “…indicates that the LSTM module is more capable of benefiting from the information embedded in the soft labels provided by the teacher model, as well as its superiority in encoding sequential inputs (Tan et al. (2015)), (Bahdanau et al. (2014)) • Best performance is achieved with temperatures > 1.”
- Enhance word representation for out-of-vocabulary on Ubuntu dialogue corpus (2018, methodology): “…MV-LSTM (Wan et al., 2016; Wu et al., 2017) 0.348 0.498 0.538 Match-LSTM (Wang & Jiang, 2015; Wu et al., 2017) 0.345 0.500 0.537 Attentive-LSTM (Tan et al., 2015; Wu et al., 2017) 0.331 0.495 0.523 Multi-Channel (Wu et al., 2017) 0.349 0.506 0.543 SMN dynamic (Wu et al., 2017) 0.397 0.529…”
- Deep Relevance Ranking Using Enhanced Document-Query Interactions (2018, background): “Passage scoring for question answering (Surdeanu et al., 2008) ranks passages by their relevance to the question; several deep networks have been proposed, e.g., Tan et al. (2015).”
- Beyond [CLS] through Ranking by Generation (2020, methodology): “…and retrieval of semantically related questions have focused primarily on discriminative methods using neural networks that learn a similarity function to compare questions and candidate answers (Severyn and Moschitti, 2015; dos Santos et al., 2015; Tan et al., 2016; Tay et al., 2017, 2018).”
- Deep Code Search (2018, background): “One can also embed two sentences (a question sentence and an answer sentence) and compare their vectors for answer selection [21, 71].”
- SelQA: A New Benchmark for Selection-Based Question Answering (2016, methodology): “Our recurrent neural network model with attention mechanisms is based on established state-of-the-art systems for answer sentence selection (Tan et al., 2015; Santos et al., 2016).”
- Denoising Bodies to Titles: Retrieving Similar Questions with Recurrent Convolutional Models (2015, background): “For instance, Feng et al. (2015) explore different similarity measures beyond cosine similarity, and Tan et al. (2015) adopt the neural attention mechanism over RNNs to generate better answer representations given the questions as context.”
- LSDSem 2017: Exploring Data Generation Methods for the Story Cloze Test (2017, methodology): “Following previous work on using BiLSTMs in NLP tasks (Tan et al., 2015; Tan et al., 2016; dos Santos et al., 2016), we chose a dimensionality of 141 for the LSTM output vectors and hidden layer H.”
- Review-based Question Generation with Adaptive Instance Transfer and Augmentation (2019, background): “QA-LSTM (Tan et al., 2016), one of the state-of-the-art QA matching models, has good generalization ability for nonfactoid questions.”
- Smart Cities: A Survey on Data Management, Security, and Enabling Technologies (2017, background): “More on LSTM can be found in [223] and [224].”
- Modeling Multi-turn Conversation with Deep Utterance Aggregation (2018, neutral): “Our baselines include: • Single-turn matching models: Basic models in (Kadlec et al., 2015; Lowe et al., 2015), including TF-IDF, CNN, RNN, LSTM and biLSTM ; We also explore other advanced single-turn matching models, MV-LSTM (Wan et al., 2016), Match-LSTM (Wang and Jiang, 2015), Attentive-LSTM (Tan”
- Alignment over Heterogeneous Embeddings for Question Answering (2019, background): “Most use stacked architectures usually coupled with attention mechanisms (He and Lin, 2016; Yin et al., 2015; Seo et al., 2016; Xiong et al., 2016b; Kumar et al., 2016; Tan et al., 2015; Wang et al., 2017a; Chen et al., 2016; Cheng et al., 2016; Golub and He, 2016).”