GooAQ: Open Question Answering with Diverse Answer Types

By Daniel Khashabi, Amos Ng, Tushar Khot, Ashish Sabharwal, Hannaneh Hajishirzi, Chris Callison-Burch. 2021

73 citations recorded by the source corpus.

While day-to-day questions come with a variety of answer types, the current question-answering (QA) literature has failed to adequately address the answer diversity of questions. To this end, we present GooAQ, a large-scale dataset with a variety of answer types. This dataset contains over 5 million questions and 3 million answers collected from Google. GooAQ questions are collected semi-automatically from the Google search engine using its autocomplete feature. This results in naturalistic questions of practical interest that are nonetheless short and expressed using simple language. GooAQ answers are mined from Google's responses to our collected questions, specifically from the answer boxes in the search results. This yields a rich space of answer types, containing both textual answers (short and long) as well as more structured ones such as collections. We benchmarkT5 models on GooAQ and observe that: (a) in line with recent work, LM's strong performance on GooAQ's short-answer questions heavily benefit from annotated data; however, (b) their quality in generating coherent and accurate responses for questions requiring long responses (such as 'how' and 'why' questions) is less reliant on observing annotated data and mainly supported by their pre-training. We release GooAQ to facilitate further research on improving QA with diverse response types.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

Survey of Hallucination in Natural Language Generation (2022, background): “It is an important task since many of the everyday questions that humans deal with and pose to search engines require in-depth explanations [81] (e.”
Task-aware Retrieval with Instructions (2022, background): “For GooAQ-technical, we sample 1,000 GooAQ questions whose answers are from stackoverflow.com .”
Nomic Embed: Training a Reproducible Long Context Text Embedder (2024, unknown): “(Lewis et al., 2021b) 52,953,088 0.23 Amazon Reviews (Ni et al., 2019) 38,682,624 0.16 S2ORC Title Abstract (Lo et al., 2020) 35438592 0.15 WikiAnswers (Fader et al., 2014) 9,912,320 0.04 S2ORC Citation Titles (Lo et al., 2020) 7,585,792 0.03 S2ORC Abstract Citation (Lo et al., 2020) 7,503,872 0.03 ”
Learning to Solve Complex Tasks by Talking to Agents (2021, background): “Given the presence of existing datasets with similar variety of answer types (Dua et al., 2019; Khashabi et al., 2021) and models that can answer questions across multiple formats (Khashabi et al., 2020), such agents are reasonable to expect and capture a broad class of QA agents.”
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction (2021, methodology): “In addition, we introduce a new benchmark for LongTail Topic-stratified Evaluation for IR, dubbed LoTTE, that features 12 domain-specific search tests, spanning StackExchange communities and using queries from GooAQ (Khashabi et al., 2021).”
CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training (2021, methodology): “356 GooAQ (Khashabi et al., 2021) contains semi- 357 automatically extracted question-answer pairs from 358 the Google question auto-complete feature.”
Read before Generate! Faithful Long Form Question Answering with Machine Reading (2022, background): “It is important since many of the everyday questions that humans deal with and pose to search engines require multisentence explanations (Khashabi et al., 2021) (e.”
UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training (2022, background): “Another open direction is whether one can build unified models that can produce answers at different granularity (Khashabi et al., 2021).”