Deep Learning for Answer Sentence Selection

By Lei Yu, Karl Moritz Hermann, Phil Blunsom, Stephen G. Pulman. 2014

421 citations recorded by the source corpus.

Answer sentence selection is the task of identifying sentences that contain the answer to a given question. This is an important problem in its own right as well as in the larger context of open domain question answering. We propose a novel approach to solving this task via means of distributed representations, and learn to match questions with answers by considering their semantic encoding. This contrasts prior work on this task, which typically relies on classifiers with large numbers of hand-crafted syntactic and semantic features and various external resources. Our approach does not require any feature engineering nor does it involve specialist linguistic data, making this model easily applicable to a wide range of domains and languages. Experimental results on a standard benchmark dataset from TREC demonstrate that---despite its simplicity---our model matches state of the art performance on the answer sentence selection task.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge (2019, methodology): “In both variations we return the answer with highest LM probability. c QAB ILINEAR This model, propsed by Yu et al. (2014) for QA, scores an answer a i with a bilinear model: qW a i , where the question q and answers a i are the average pre-trained word embeddings and W is a learned parameter…”
Coarse-to-Fine Question Answering for Long Documents (2016, methodology): “Our model takes a hierarchical approach (see Figure 1), where ﬁrst a fast model is used to select a few sentences from the document that are relevant for answering the question (Yu et al., 2014; Yang et al., 2016a).”
Webly Supervised Joint Embedding for Cross-Modal Image-Text Retrieval (2018, background): “Averaged word vectors has been shown to be a strong feature for text in several tasks [26, 27, 58].”
WikiQA: A Challenge Dataset for Open-Domain Question Answering (2015, methodology): “, 2013) and sentence semantic models (Yu et al., 2014).”
The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems (2015, methodology): “Work on unstructured dialogues, recently pi-oneered by Ritter et al. [22], proposed a response generation model for Twitter data based on ideas from Statistical Machine Translation.”
A Deep Reinforcement Learning Chatbot (2017, methodology): “In principle, it is possible to compute input features by encoding the dialogue context and candidate response using Recurrent Neural Networks (RNNs) or Convolutional Neural Networks (Con-vNets) (Socher et al. 2013, Blunsom et al. 2014, Cho et al. 2014, Yu et al. 2014, Kiros et al. 2015).”
Neural Variational Inference for Text Processing (2015, methodology): “To facilitate direct comparison with previous work we follow the same experimental setup as Yu et al. (2014) and Severyn (2015).”
Recent advances in convolutional neural networks (2015, methodology): “[338] apply one convolutional layer to model the sentence, while Kalchbrenner et al .”
Neural Semantic Encoders (2016, neutral): “While the LSTMs and LSTMs attention models outperform the [23] 0.5993 0.6068 Paragraph Vector [24] 0.5110 0.5160 Bigram-CNN [25] 0.6190 0.6281 Three-layer LSTMs [26] 0.6552 0.6747 Three-layer LSTMs attention [26] 0.6639 0.6828 NASM [26] 0.6705 0.6914 MMA-NSE attention 0.6811 0.6993 Table 3: Test…”
ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs (2015, methodology): “(iv) PV: Paragraph Vector (Le and Mikolov, 2014); (v) CNN: bigram convolutional neural network (Yu et al., 2014); (vi) PV-Cnt: combine PV with (i) and (ii); (vii) CNN-Cnt: combine CNN with (i) and (ii).”
Question Answering over Freebase with Multi-Column Convolutional Neural Networks (2015, neutral): “Yu et al. (2014) proposed a bigram model based on convolutional neural networks to select answer sentences from text data.”
Attentive Pooling Networks (2016, methodology): “For example, in (Yu et al., 2014; Feng et al., 2015; Severyn & Moschitti, 2015), the authors generate the representations of questions and answers separately, and score a QA pair using a similarity metric on top of these representations.”
Pairwise Word Interaction Modeling with Deep Neural Networks for Semantic Similarity Measurement (2016, methodology): “…al., 2015) 0.5099 0.5132 PV (Le and Mikolov, 2014) 0.5110 0.5160 PV-Cnt (Yang et al., 2015) 0.5976 0.6058 LCLR (Yih et al., 2013) 0.5993 0.6086 CNN (Yu et al., 2014) 0.6190 0.6281 CNN-Cnt (Yang et al., 2015) 0.6520 0.6652 He et al. (2015) 0.6930 0.7090 This work 0.7090 0.7234 10 Analysis Ablation…”
Deconvolutional Latent-Variable Model for Text Sequence Matching (2017, background): “In this regard, text sequence matching can be viewed as either a binary or multiclass classification problem (Yu et al. 2014).”
Siamese CBOW: Optimizing Word Embeddings for Sentence Representations (2016, positive): “Surprisingly, simply averaging word embeddings of all words in a text has proven to be a strong baseline or feature across a multitude of tasks (Faruqui et al., 2014; Yu et al., 2014; Gershman and Tenenbaum, 2015; Kenter and de Rijke, 2015).”
Self-Training for Jointly Learning to Ask and Answer Questions (2018, methodology): “Treating QA as an answer sentence selection task is quite common in literature (e.g. see Yu et al., 2014).”
Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications (2019, methodology): “Baselines We compare our approach to the following baselines: CNN + LR (Yu et al., 2014b) using unigrams and bigrams, CNN (Severyn and Moschitti, 2015) using additional bilinear similarity features, CNTN (Qiu and Huang, 2015) using neural tensor network, LSTM (Tay et al., 2017) us-ing single and…”
Multichannel Variable-Size Convolution for Sentence Classification (2015, methodology): “Collobert and Weston (2008) and Yu et al. (2014) used one layer of convolution over phrases detected by a sliding window on a target sentence, then used max- or average-pooling to form a sentence representation.”