Attention-Based Convolutional Neural Network for Machine Comprehension

By Wenpeng Yin, Sebastian Ebert, Hinrich Schütze. 2016

101 citations recorded by the source corpus.

Understanding open-domain text is one of the primary challenges in natural language processing (NLP). Machine comprehension benchmarks evaluate the system's ability to understand text based on the text content only. In this work, we investigate machine comprehension on MCTest, a question answering (QA) benchmark. Prior work is mainly based on feature engineering approaches. We come up with a neural network framework, named hierarchical attention-based convolutional neural network (HABCNN), to address this task without any manually designed features. Specifically, we explore HABCNN for this task by two routes, one is through traditional joint modeling of passage, question and answer, one is through textual entailment. HABCNN employs an attention mechanism to detect key phrases, key sentences and key snippets that are relevant to answering the question. Experiments show that HABCNN outperforms prior deep learning approaches by a big margin.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Model Architectures

Citations in the corpus

Option Comparison Network for Multiple-choice Reading Comprehension (2019, methodology): “…development of deep learning, various neural models have been proposed for MCRC and achieve promising re-sults in recent years (Chen et al., 2016; Yin et al., 2016; Trischler et al., 2016; Dhingra et al., 2017; Tay et al., 2018; Parikh et al., 2018; Zhu et al., 2018; Wang et al., 2018; Xu et…”
Machine Comprehension Using Match-LSTM and Answer Pointer (2016, background): “…of applying neural network models in NLP, there has been much interest in building end-to-end neural architectures for various NLP tasks, including several pieces of work on machine comprehension (Hermann et al., 2015; Hill et al., 2016; Yin et al., 2016; Kadlec et al., 2016; Cui et al., 2016).”
Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering (2016, background): “…tensor networks (Socher et al., 2013), recursive networks (Iyyer et al., 2014), convolution neural networks (Yih et al., 2014; Dong et al., 2015; Yin et al., 2015), attention models (Hermann et al., 2015; Yin et al., 2015; Santos et al., 2016), and memories (Graves et al., 2014; Weston et al.,…”
A Co-Matching Model for Multi-choice Reading Comprehension (2018, methodology): “…on pairwise sequence matching, where either the passage is matched against the sequence that concatenates both the question and a candidate answer (Yin et al., 2016), or the passage is matched against the question alone followed by a second step of selecting an answer using the matching result of…”
A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Dataset (2018, methodology): “Recent techniques such as Wang et al. (2015) and Yin et al. (2016) have performed well on this dataset.”
A Parallel-Hierarchical Model for Machine Comprehension on Sparse Data (2016, methodology): “Models designed specifically for MCTest include those of Richardson et al. (2013), and more recently Sachan et al. (2015), Wang and McAllester (2015), and Yin et al. (2016).”
Learning What is Essential in Questions (2017, neutral): “For instance, Neural Networks based models (Wang et al., 2016; Tymoshenko et al., 2016; Yin et al., 2016) implicitly compute some kind of attention.”
TwoWingOS: A Two-Wing Optimization Strategy for Evidential Claim Verification (2018, background): “Open-domain question answering (QA) against a text corpus (Yin et al., 2016; Chen et al., 2017; Wang et al., 2018) can also be treated as claim verification problem, if we treat (question, correct answer) as a claim.”
MITRE at SemEval-2018 Task 11: Commonsense Reasoning without Commonsense Knowledge (2018, background): “Yin et al. (2016) restate the question and answer and incorporate a question-type classiﬁer.”
Answering Science Exam Questions Using Query Rewriting with Background Knowledge (2018, methodology): “Recent techniques such as those presented by Wang et al. (2015) and Yin, Ebert, and Schütze (2016) have performed well on this dataset.”
Hierarchical Attention Flow for Multiple-Choice Reading Comprehension (2018, methodology): “Inspired by Yin, Ebert, and Schütze (2016) and Trischler et al. (2016), we present the neural network based hierarchical attention flow, depicted in Figure 2, to adequately leverage candidate options to model the word level and sentence level interactions among passages, questions and candidate…”
Prerequisite Skills for Reading Comprehension: Multi-Perspective Analysis of MCTest Datasets and Systems (2017, methodology): “It should be noted that the performance of the system we report in this paper is incompatible with the one of the original paper (Yin, Ebert, and Schütze 2016): this might be contributed to the difference of the data used for the evaluation (test vs. development).”
Text feature extraction based on deep learning: a review (2017, background): “…such as semantic parsing [43], information retrieval [44, 45], semantic role labeling [46, 47], sentimental analysis [48], question answering [49 – 52], machine translation [53 – 56], text classification [57], summarization [58, 59], and text generation [60], as well as information extraction,…”