Frustratingly Hard Evidence Retrieval for QA Over Books

By Xiangyang Mou, Mo Yu, Bingsheng Yao, Chenghao Yang, Xiaoxiao Guo, Saloni Potdar, Hui Su. 2020

12 citations recorded by the source corpus.

A lot of progress has been made to improve question answering (QA) in recent years, but the special problem of QA over narrative book stories has not been explored in-depth. We formulate BookQA as an open-domain QA task given its similar dependency on evidence retrieval. We further investigate how state-of-the-art open-domain QA approaches can help BookQA. Besides achieving state-of-the-art on the NarrativeQA benchmark, our study also reveals the difficulty of evidence retrieval in books with a wealth of experiments and analysis - which necessitates future effort on novel solutions for evidence retrieval in BookQA.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Shiyu Chang
2. Mohit Bansal
3. Mo Yu
4. Jan Leike
5. Jeff Wu
6. Sergey Levine
7. Ryan J. Lowe
8. Daniel M. Ziegler

In Information Systems

1. Chuang Gan
2. Paul Christiano
3. Xiaoxiao Guo
4. Daniel M. Ziegler
5. Ryan J. Lowe
6. Mo Yu
7. Long Ouyang
8. Shiyu Chang

In Natural Language Processing

1. Shiyu Chang
2. Mo Yu
3. Xiaoxiao Guo
4. Murray Campbell
5. Chuang Gan
6. Paul Christiano
7. Daniel M. Ziegler
8. Ryan J. Lowe

In Question Answering & Reading Comprehension

1. Chuang Gan
2. Mo Yu

Citations in the corpus

• Recursively Summarizing Books with Human Feedback (2021, background): “SummaQA requires paragraph-aligned summaries, which we do not have, and so we report results on ROUGE and BERTScore.”
• RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (2024, unknown): “(Mou et al., 2020) 15 .”
• ReadTwice: Reading Very Large Documents with Memories (2021, unknown): “Table 2 contrasts R EAD T WICE to other methods on extremely large contexts: BiDAF (Kociský et al., 2018), R 3 (Wang et al., 2018), BM25 + BERT Reader / Ranker (Mou et al., 2020) and our own implementation of R O BERT A and ETC 5 .”
• Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning (2020, methodology): “The techniques above have been applied to deal with evidence from Wikipedia, news collections, and, recently, books (Mou et al., 2020).”