BioTABQA: Instruction Learning for Biomedical Table Question Answering
By Man Luo, Sharad Saxena, Swaroop Mishra, Mihir Parmar, Chitta Baral. 2022
19 citations recorded by the source corpus.
Table Question Answering (TQA) is an important but under-explored task. Most of the existing QA datasets are in unstructured text format and only few of them use tables as the context. To the best of our knowledge, none of TQA datasets exist in the biomedical domain where tables are frequently used to present information. In this paper, we first curate a table question answering dataset, BioTABQA, using 22 templates and the context from a biomedical textbook on differential diagnosis. BioTABQA can not only be used to teach a model how to answer questions from tables but also evaluate how a model generalizes to unseen questions, an important scenario for biomedical applications. To achieve the generalization evaluation, we divide the templates into 17 training and 5 cross-task evaluations. Then, we develop two baselines using single and multi-tasks learning on BioTABQA. Furthermore, we explore instructional learning, a recent technique showing impressive generalizing performance. Experimental results show that our instruction-tuned model outperforms single and multi-task baselines on an average by ~23% and ~6% across various evaluation settings, and more importantly, instruction-tuned model outperforms baselines by ~5% on cross-tasks.
Subfields: Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
- Chitta Baral
- Hannaneh Hajishirzi
- Swaroop Mishra
- Daniel Khashabi
- Noah A. Smith
- Mihir Parmar
- Yejin Choi
- Luke S. Zettlemoyer
Citations in the corpus
- Self-Instruct: Aligning Language Models with Self-Generated Instructions (2022, background): “A series of works have found evidence that vanilla language models can be effective at following general language instructions if tuned with annotated “instructional” data – datasets containing language instructional commands and their desired outcome based on human judgement (Weller et al., 2020; M”
- Is a Question Decomposition Unit All We Need? (2022, background): “…instructional prompts have been pivotal in improving the performance of LMs and achieving zero-shot generalization (Mishra et al., 2021b; Wei et al., 2021; Sanh et al., 2021b; Wei et al., 2022; Ouyang et al., 2022; Parmar et al., 2022; Puri et al., 2022; Kuznia et al., 2022; Luo et al., 2022).”
- How Many Data Samples is an Additional Instruction Worth? (2022, methodology): “In addition, many approaches have been proposed to improve model performance using instructions (Wu et al., 2022; Lin et al., 2021; Wang et al., 2022b; Luo et al., 2022; Kuznia et al., 2022; Patel et al., 2022; Mishra and Nouri, 2022).”
- HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Models (2022, background): “, 2021), tabular question answering (Luo et al., 2022), relation extraction (Chen et al.”