Improved Representation Learning for Question Answer Matching

By Ming Tan, C. D. Santos, Bing Xiang, Bowen Zhou. 2016

254 citations recorded by the source corpus.

Passage-level question answer matching is a challenging task since it requires effective representations that capture the complex semantic relations between questions and answers. In this work, we propose a series of deep learning models to address passage answer selection. To match passage answers to questions accommodating their complex semantic relations, unlike most previous work that utilizes a single deep learning structure, we develop hybrid models that process the text using both convolutional and recurrent neural networks, combining the merits on extracting linguistic information from both structures. Additionally, we also develop a simple but effective attention mechanism for the purpose of constructing better answer representations according to the input question, which is imperative for better modeling long answer sequences. The results on two public benchmark datasets, InsuranceQA and TREC-QA, show that our proposed models outperform a variety of strong baselines.

Subfields: Artificial Intelligence, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

A Structured Self-attentive Sentence Embedding (2017, background): “, 2014; dos Santos & Gatti, 2014; Kim, 2014) as an intermediate step in creating sentence representations to solve a wide variety of tasks including classification and ranking (Yin & Schütze, 2015; Palangi et al., 2016; Tan et al., 2016; Feng et al., 2015).”
Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications (2019, methodology): “Since the attention mechanism naturally explores relevancy, it has been widely used in QA models to relate the question to candidate answers (Tan et al., 2016; Santos et al., 2016; Sha et al., 2018).”
TutorialVQA: Question Answering Dataset for Tutorial Videos (2019, methodology): “Baseline2 is based on the attentive LSTM (Tan et al., 2016), which has been developed for the InsuranceQA task.”
Deep Learning--based Text Classification (2020, methodology): “toid answer selection [118], Hyperbolic representation learning [119], and question-answering using a deep similarity neural network [120]. Fig. 17. The architectures of the Siamese models studied in [111]. 2.10 Hybrid Models Many Hybrid models have been developed to combine LSTM and CNN architectur”
LSDSem 2017: Exploring Data Generation Methods for the Story Cloze Test (2017, methodology): “Following previous work on using BiLSTMs in NLP tasks (Tan et al., 2015; Tan et al., 2016; dos Santos et al., 2016), we chose a dimensionality of 141 for the LSTM output vectors and hidden layer H.”
Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learning (2018, result): “Over the past few years, deep learning has yielded superior results to traditional retrieval techniques in this domain [51, 52, 66].”
Knowledge-aware Attentive Neural Network for Ranking Question Answer Pairs (2018, background): “Instead of learning the representations of the question and the answer separately, some recent studies exploit attention mechanisms to learn the interaction information between questions and answers, which can better focus on relevant parts of the input [2, 4, 7].”
A Compare-Aggregate Model for Matching Text Sequences (2016, methodology): “Attentive network: Soft-attention mechanism (Bahdanau et al., 2014) has been widely used for sequence matching in machine comprehension (Hermann et al., 2015), text entailment (Rockt¨aschel et al., 2015) and question answering (Tan et al., 2016).”
MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale (2020, methodology): “Similar sequence lengths are commonly used on the benchmarks that we study (e.g., Mass et al., 2019; Tan et al., 2016).”
Cross-topic Argument Mining from Heterogeneous Sources (2018, background): “Similar approaches have been shown to achieve state-of-the-art results in aspect-based sentiment analysis (Wang et al., 2016), question answering (Tan et al., 2016), and discourse parsing (Li et al., 2016).”
A Survey on Text Classification: From Traditional to Deep Learning (2020, methodology): “[196] introduce CNN and RNN and generate answer embeddings by using a simple one-way attention mechanism affected through the question context.”
A Survey on Text Classification: From Shallow to Deep Learning (2020, methodology): “To catch the complicated semantic relationship among each question and candidate answers for the QA task, Tan et al. [154] introduce CNN and RNN and generate answer embeddings by using a simple one-way attention mechanism affected through the question context.”
Advances and Challenges in Conversational Recommender Systems: A Survey (2021, methodology): “Then, a matching function is used to combine the two representations and output a matching probability [63, 164, 139, 45, 175].”