Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering

By Peng Li, Wei Li, Zhen He, Xuguang Wang, Ying Cao, Jie Zhou, Wei Xu. 2016

104 citations recorded by the source corpus.

While question answering (QA) with neural network, i.e. neural QA, has achieved promising results in recent years, lacking of large scale real-word QA dataset is still a challenge for developing and evaluating neural QA system. To alleviate this problem, we propose a large scale human annotated real-world QA dataset WebQA with more than 42k questions and 556k evidences. As existing neural QA methods resolve QA either as sequence generation or classification/ranking problem, they face challenges of expensive softmax computation, unseen answers handling or separate candidate answer generation component. In this work, we cast neural QA as a sequence labeling problem and propose an end-to-end sequence labeling model, which overcomes all the above challenges. Experimental results on WebQA show that our model outperforms the baselines significantly with an F1 score of 74.69% with word-based input, and the performance drops only 3.72 F1 points with more challenging character-based input.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension (2017, methodology): “, 2015) for answer sentence selection, and the Chinese language WebQA (Li et al., 2016) dataset, which focuses on the task of answer phrase extraction.”
PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation (2021, background): “Closed-book question answering (QA) tasks, including WebQA [38].”
Learning Recurrent Span Representations for Extractive Question Answering (2016, methodology): “In this section, we motivate and describe the architecture for R A S O R in a top-down manner.”
Gated-Attention Readers for Text Comprehension (2016, background): “The intuition behind this computation is that multiple hops over the same set of memories allow the model to reason over them individually.”
Making Neural QA as Simple as Possible but not Simpler (2017, background): “Feeding additional word-level features similar to ours is rarely done with the exception of Wang et al. (2017); Li et al. (2016).”
Automatic Rule Extraction from Long Short Term Memory Networks (2016, methodology): “A recent line of work Li et al. (2016) Hewlett et al. (2016) Rajpurkar et al. (2016) Miller et al. (2016) has focused on neural network techniques for extracting answers directly from documents.”
Embracing data abundance: BookTest Dataset for Reading Comprehension (2016, background): “…from the research community (Kobayashi et al., 2016; Kadlec et al., 2016; Chen et al., 2016; Sordoni et al., 2016; Dhingra et al., 2016; Trischler et al., 2016b; Weis-senborn, 2016; Cui et al., 2016b; Cui et al., 2016a; Li et al., 2016) with a new state-of-the-art model coming out every few weeks.”
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation (2021, neutral): “• Closed-Book Question Answering: NLPCC-DBQA 12, CHIP2019, cMedQA [59], cMedQA2 [60], CKBQA 13, WebQA [61].”
A Sentence Cloze Dataset for Chinese Machine Reading Comprehension (2020, background): “, 2016) News, Story Cloze Word 28K 100K WebQA (Li et al., 2016) Web NQ Entity - 42K CMRC 2017 (Cui et al.”
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation (2021, unknown): “• Closed-Book Question Answering (CB-QA) : NLPCC-DBQA 16 , CHIP2019, cMedQA [78], cMedQA2 [79], CKBQA 17 , WebQA [80].”
FreebaseQA: A New Factoid QA Data Set Matching Trivia-Style Question-Answer Pairs with Freebase (2019, background): “Ad-ditionally, many similar factoid QA data sets are also released for other non-English languages, e.g. WebQA in (Li et al., 2016).”
A Comprehensive Overview of Large Language Models (2023, unknown): “0 controllable dataset NLU: NLPCC2014-SC, SE-ABSA16_PHNS, SE-ABSA16_CAME, BDCI2019, EPRSTMT [293], COTE-BD [268], COTE-MFW [268], OCNLI [269], CMNLI [269], OCNLI-FC [293], CLUEWSC [269] CLUEWSC-FC [293], FinRE [270], SanWen [271], AFQMC [269], LCQMC [242], PAWS-X [231], BQ Corpus [272], CSL [269] CS”
Joint Training of Candidate Extraction and Answer Selection for Reading Comprehension (2018, methodology): “The similar training strategy is commonly employed when RL process is involved (Ranzato et al., 2015; Li et al., 2016a; Hu et al., 2018).”
Dual Attention Network for Product Compatibility and Function Satisfiability Analysis (2017, unknown): “…in NLP tasks, such as machine translation(Bahdanau, Cho, and Bengio 2014), sentence summarization (Rush, Chopra, and Weston 2015), sentiment analysis (Tang, Qin, and Liu 2016), question and answering (Li et al. 2016) and reading comprehension (Kumar et al. 2015; Xiong, Zhong, and Socher 2016).”
FastQA: A Simple and Efficient Neural Architecture for Question Answering (2017, methodology): “Li et al. (2016), on the other hand, successfully employed two simple binary features in addition to word-embeddings during encoding which is similar to our encoder.”
Are BLEU and Meaning Representation in Opposition? (2018, methodology): “We adopt an alternative approach, which is to use inner attention 1 (Liu et al., 2016; Li et al., 2016) to compute several weighted averages of the encoder states (Lin et al., 2017).”