Question Answering through Transfer Learning from Large Fine-grained Supervision Data

By Sewon Min, Minjoon Seo, Hannaneh Hajishirzi. 2017

121 citations recorded by the source corpus.

We show that the task of question answering (QA) can significantly benefit from the transfer learning of models trained on a different large, fine-grained QA dataset. We achieve the state of the art in two well-studied QA datasets, WikiQA and SemEval-2016 (Task 3A), through a basic transfer learning technique from SQuAD. For WikiQA, our model outperforms the previous best model by more than 8%. We demonstrate that finer supervision provides better guidance for learning lexical and syntactic information than coarser supervision, through quantitative results and visual analysis. We also show that a similar transfer learning procedure achieves the state of the art on an entailment task.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Universal Language Model Fine-tuning for Text Classification (2018, background): “Fine-tuning Fine-tuning has been used successfully to transfer between similar tasks, e.g. in QA (Min et al., 2017), for distantly supervised sentiment analysis (Severyn and Moschitti, 2015), or MT domains (Sennrich et al., 2015) but has been shown to fail between unrelated ones (Mou et al., 2016).”
Simple and Effective Multi-Paragraph Reading Comprehension (2017, methodology): “Our approach to allowing a reading comprehension model to produce a per-paragraph no-answer score is related to the approach used in the BiDAF-T (Min et al., 2017) model to produce per-sentence classiﬁcation scores, although we use an attention-based method instead of max-pooling.”
Learned in Translation: Contextualized Word Vectors (2017, background): “Attention mechanisms have also been successfully applied to NLP tasks like entailment [Conneau et al., 2017], summarization [Nallapati et al., 2016], question answering [Seo et al., 2017, Xiong et al., 2017, Min et al., 2017], and semantic parsing [Dong and Lapata, 2016].”
The Natural Language Decathlon: Multitask Learning as Question Answering (2018, methodology): “This boosts performance on both MNLI and MWSC above prior baselines.”
Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences (2018, methodology): “As a neural baseline, we apply this solver by Seo et al. (2017), which was originally proposed for SQuAD but has been shown to generalize well to another domain (Min et al., 2017).”
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning (2019, background): “Recent studies (Howard and Ruder, 2018; Min et al., 2017; Devlin et al., 2018) have shown the beneﬁt of ﬁne-tuning on similar tasks or datasets for knowledge transfer.”
Efficient and Robust Question Answering from Minimal Context over Documents (2018, background): “Sentence selection The task of selecting sentences that can answer to the question has been studied across several QA datasets (Yang et al., 2015), by modeling relevance between a sentence and the question (Yin et al., 2016; Miller et al., 2016; Min et al., 2017).”
Errudite: Scalable, Reproducible, and Testable Error Analysis (2019, background): “We surveyed 10 papers with error analyses that examine a sample of incorrect predictions, e.g., (Wadhwa et al., 2018; Min et al., 2017) 1 , and found the sample sizes ranged from 50 to 200 model errors ( µ = 85 .”
MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension (2019, background): “Nevertheless, there has been relatively little work (Min et al., 2017; Chung et al., 2018; Sun et al., 2018) that explores the relations between the different datasets, including whether a model trained on one dataset generalizes to another.”
Improving Machine Reading Comprehension with General Reading Strategies (2018, positive): “Transfer learning techniques have been successfully applied to machine reading comprehension (Golub et al., 2017; Chung et al., 2018) and question answering (Min et al., 2017; Wiese et al., 2017).”
Question Answering as Global Reasoning Over Semantic Abstractions (2018, methodology): “We use the recent deep learning reading comprehension model of Seo et al. (2016), which is one of the top performing systems on the SQuAD dataset and has been shown to generalize to another domain as well (Min, Seo, and Hajishirzi 2017).”
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning (2020, background): “Recent studies (Min et al., 2017; Howard & Ruder, 2018; Huang et al., 2019; Jin et al., 2019) have shown the effectiveness of pre-training on similar tasks or datasets then fine-tuning on the target dataset for transfer learning.”
Cross-Lingual Transfer Learning for Question Answering (2019, methodology): “Min et al. (2017) use source data to pre-train QA model, and then ﬁne-tune the model with target data.”
Multi-Stage Pretraining for Low-Resource Domain Adaptation (2020, background): “In cases where the target task has limited labeled data, prior work has also employed transfer learning by pre-training on a source dataset with abundant labeled data before ﬁne-tuning on the target task dataset (Min et al., 2017; Chung et al., 2018; Wiese et al., 2017).”
TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection (2019, background): “It is still worthwhile to mention that Min, Seo, and Hajishirzi (2017) explored transfer learning for the BiDAF model for MR.”
Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering (2019, background): “Note that some of the works discussed here transfer knowledge from external datasets into the QA task they address (Chung et al., 2017; Sun et al., 2018; Pan et al., 2019; Min et al., 2017; Qiu et al., 2018; Chen et al., 2017).”
Neural Speed Reading via Skim-RNN (2017, background): “, 2017) and achieves state of the art in WikiQA and SemEval-2016 (Task 3A) (Min et al., 2017).”