Adaptive Information Seeking for Open-Domain Question Answering

By Yunchang Zhu, Liang Pang, Yanyan Lan, Huawei Shen, Xueqi Cheng. 2021

43 citations recorded by the source corpus.

Information seeking is an essential step for open-domain question answering to efficiently gather evidence from a large corpus. Recently, iterative approaches have been proven to be effective for complex questions, by recursively retrieving new evidence at each step. However, almost all existing iterative approaches use predefined strategies, either applying the same retrieval function multiple times or fixing the order of different retrieval functions, which cannot fulfill the diverse requirements of various questions. In this paper, we propose a novel adaptive information-seeking strategy for open-domain question answering, namely AISO. Specifically, the whole retrieval and answer process is modeled as a partially observed Markov decision process, where three types of retrieval operations (e.g., BM25, DPR, and hyperlink) and one answer operation are defined as actions. According to the learned policy, AISO could adaptively select a proper retrieval action to seek the missing evidence at each step, based on the collected evidence and the reformulated query, or directly output the answer when the evidence set is sufficient for the question. Experiments on SQuAD Open and HotpotQA fullwiki, which serve as single-hop and multi-hop open-domain QA benchmarks, show that AISO outperforms all baseline methods with predefined strategies in terms of both retrieval and answer evaluations.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Systems, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Weizhu Chen
2. Nan Duan
3. Yelong Shen
4. Jianfeng Gao
5. Zhihong Shao
6. Zhibin Gou
7. Yeyun Gong
8. Yujiu Yang

In Computer Vision and Pattern Recognition

1. Weizhu Chen
2. Yelong Shen
3. Nan Duan
4. Zhihong Shao
5. Yeyun Gong
6. Yujiu Yang
7. Jianfeng Gao
8. Zhe Gan

In Information Systems

1. Zhihong Shao
2. Yelong Shen
3. Zhibin Gou
4. Weizhu Chen
5. Yeyun Gong
6. Yujiu Yang
7. Nan Duan

In Information Retrieval & Search

1. Yelong Shen
2. Weizhu Chen
3. Nan Duan
4. Debajyoti Chatterjee

In Natural Language Processing

1. Weizhu Chen
2. Yelong Shen
3. Nan Duan
4. Zhihong Shao
5. Zhibin Gou
6. Yeyun Gong
7. Yujiu Yang
8. Jianfeng Gao

Citations in the corpus

• ReAct: Synergizing Reasoning and Acting in Language Models (2022, neutral): “b (Zhu et al., 2021; Lewis et al., 2020) 0 5 10 15 20 #CoT-SC trials 26 28 30 32 34”
• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing (2023, methodology): “[115] Yunchang Zhu, Liang Pang, Yanyan Lan, Huawei Shen, and Xueqi Cheng.”
• LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs (2024, unknown): “…(Asai et al., 2019) 60.0 HopRetrieve (Li et al., 2021) 62.1 MDR (Xiong et al., 2020b) 62.3 HopRetrieve-plus (Li et al., 2021) 66.5 AISO (Zhu et al., 2021) 68.1 COS (Ma et al., 2023) 68.2 No Fine-tuning RAG DSP (Khattab et al., 2022) 51.4 PromptRank (Khalifa et al., 2023) 55.7 LongRAG…”