Character-Level Question Answering with Attention

By David Golub, Xiaodong He. 2016

191 citations recorded by the source corpus.

We show that a character-level encoder-decoder framework can be successfully applied to question answering with a structured knowledge base. We use our model for single-relation question answering and demonstrate the effectiveness of our approach on the SimpleQuestions dataset (Bordes et al., 2015), where we improve state-of-the-art accuracy from 63.9% to 70.9%, without use of ensembles. Importantly, our character-level model has 16x fewer parameters than an equivalent word-level model, can be learned with significantly less data compared to previous work, which relies on data augmentation, and is robust to new entities in testing.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

In Question Answering & Reading Comprehension

Citations in the corpus

Attention-based LSTM for Aspect-level Sentiment Classification (2016, background): “Neural networks have achieved state-of-the-art performance in a variety of NLP tasks such as machine translation (Lample et al., 2016), paraphrase identiﬁcation (Yin et al., 2015), question answering (Golub and He, 2016) and text summarization (Rush et al., 2015).”
Improved Neural Relation Detection for Knowledge Base Question Answering (2017, background): “Golub and He (2016) propose a generative framework for single-relation KBQA which predicts relation with a character-level sequence-to-sequence model.”
Natural Language to Structured Query Generation via Meta-Learning (2018, background): “Semantic Parsing Mapping natural language to logic forms has been actively studied in natural language processing research (Zettlemoyer and Collins, 2005; Giordani and Moschitti, 2010; Artzi and Zettlemoyer, 2011; Berant et al., 2013; Vlachos and Clark, 2014; Yih et al., 2014, 2015; Wang et al., 201”
Alignment over Heterogeneous Embeddings for Question Answering (2019, background): “Most use stacked architectures usually coupled with attention mechanisms (He and Lin, 2016; Yin et al., 2015; Seo et al., 2016; Xiong et al., 2016b; Kumar et al., 2016; Tan et al., 2015; Wang et al., 2017a; Chen et al., 2016; Cheng et al., 2016; Golub and He, 2016).”
Scale down Transformer by Grouping Features for a Lightweight Character-level Language Model (2020, background): “…the ﬁeld of natural language processing (NLP) such as classiﬁcation (Zhang et al., 2015), sequence tagging (Guo et al., 2019a), question answering (He and Golub, 2016), and scene text recognition (Baek et al., 2019; Hwang and Sung, 2016), with its simplicity on generating text and its…”
Core techniques of question answering systems over knowledge bases: a survey (2017, unknown): “QA systems F-measure Reference Bordes et al. (2015) 0.627 (Bordes et al. 2015) Yin et al. (2016) 0.683 (Yin et al. 2016) Dai et al. (2016) ⇤ 0.626 (Dai et al. 2016) Golub and He (2016) 0.709 (Golub & He 2016) Lukovnikov et al. (2017) 0.712 (Lukovnikov et al. 2017) Table 8.”
Seq2Seq and Multi-Task Learning for joint intent and content extraction for domain specific interpreters (2018, background): “Although these solutions often have better results than letter-encoding [4], we found that the later worked very well for our application [5].”
Deep Learning for Entity Matching: A Design Space Exploration (2018, background): “[27] build upon RNNs to learn representations that summarize questions and entities.”
On internal language representations in deep learning: an analysis of machine translation and speech recognition (2018, methodology): “For example, character-based neural networks have been used in a number of tasks, including machine translation [219], language modeling [167, 177], POS tagging [208, 295], speech recognition [17, 222], dialect identification [28, 174], text classification [357], and factoid question answering [126]”
Knowledge Graph Embedding Based Question Answering (2019, methodology): “There are also several work [15, 18, 27, 46] explored to leverage the character-level neural networks to advance the perfor-”
SimpleQuestions Nearly Solved: A New Upperbound and Baseline Approach (2018, unknown): “odel word embeddings are initialized with FastText (Bojanowski et al., 2016) and frozen. The Previous Work Acc. Random guess (Bordes et al., 2015) 4.9 Memory NN (Bordes et al., 2015) 61.6 Attn. LSTM (He and Golub, 2016) 70.9 GRU (Lukovnikov et al., 2017) 71.2 BiGRU-CRF & BiGRU 73.7 (Mohammed et ”
Neural, symbolic and neural-symbolic reasoning on knowledge graphs (2021, result): “Other researches (Dai et al., 2016; Golub and He, 2016; Mohammed et al., 2018; Türe and Jojic, 2017; Yin et al., 2016) are the similar neural reasoning methods for single-relation KGQA.”