Frustratingly Easy Natural Question Answering

By Lin Pan, Rishav Chakravarti, Anthony Ferritto, Michael R. Glass, Alfio Massimiliano Gliozzo, Salim Roukos, Radu Florian, Avirup Sil. 2019

14 citations recorded by the source corpus.

Existing literature on Question Answering (QA) mostly focuses on algorithmic novelty, data augmentation, or increasingly large pre-trained language models like XLNet and RoBERTa. Additionally, a lot of systems on the QA leaderboards do not have associated research documentation in order to successfully replicate their experiments. In this paper, we outline these algorithmic components such as Attention-over-Attention, coupled with data augmentation and ensembling strategies that have shown to yield state-of-the-art results on benchmark datasets like SQuAD, even achieving super-human performance. Contrary to these prior results, when we evaluate on the recently proposed Natural Questions benchmark dataset, we find that an incredibly simple approach of transfer learning from BERT outperforms the previous state-of-the-art system trained on 4 million more examples than ours by 1.9 F1 points. Adding ensembling strategies further improves that number by 2.3 F1 points.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Colin Raffel
2. Mohit Bansal
3. Adam Roberts
4. Jianfeng Gao
5. Nan Duan
6. Yejin Choi
7. Luke S. Zettlemoyer
8. Hannaneh Hajishirzi

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Colin Raffel
3. Mohit Bansal
4. Jianfeng Gao
5. Dacheng Tao
6. Nan Duan
7. Trevor Darrell
8. Cordelia Schmid

In Natural Language Processing

1. Colin Raffel
2. Nan Duan
3. Adam Roberts
4. Noam Shazeer
5. Mohit Bansal
6. Daxin Jiang
7. Yejin Choi
8. Hannaneh Hajishirzi

In Question Answering & Reading Comprehension

1. Nan Duan
2. Daxin Jiang

Citations in the corpus

• How Much Knowledge Can You Pack into the Parameters of a Language Model? (2020, background): “9 from Pan et al. (2019) 4 but out-performs the best baseline published alongside the dataset (recall of 33 .”
• RikiNet: Reading Wikipedia Pages for Natural Question Answering (2020, methodology): “For the second challenge, unlike prior works on NQ dataset (Alberti et al., 2019b; Pan et al., 2019) that only predict the short answer and directly select its paragraph as long answer, RikiNet employs a multi-level cascaded answer predictor which jointly predict the short answer span, the long…”