Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension

By Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, Hannaneh Hajishirzi. 2017

387 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Multimodal Learning, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Multimodal Learning

In Question Answering & Reading Comprehension

Citations in the corpus

Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering (2018, background): “ing the most useful pieces of knowledge remains an open challenge, our ‘oracle’ experiments with the fact fused while generating a question qand an interpretation (by 2Only ˘5% of the TQA questions ofKembhavi et al. (2017) require additional common knowledge. 3To avoid ambiguity in the term ‘human p”
Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences (2018, methodology): “Since BiDAF was designed for cloze style questions, we apply it to our multiple-choice setting following the procedure by Kembhavi et al. (2017): Specifically, we score each answer-option by computing the similarity value of it’s output span with each of the candidate answers, computed by phrasal…”
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge (2018, methodology): “al., 2016), to our task. As BiDAF is a direct answer system, we adapted it to multiple-choice QA following the approach used in several previous projects (Khashabi et al., 2018b; Welbl et al., 2017a; Kembhavi et al., 2017) as follows: First, given a question, we create a single paragraph by concaten”
The Power of Scale for Parameter-Efficient Prompt Tuning (2021, background): “The out-of-domain datasets are TextbookQA (Kembhavi et al., 2017), RACE (Lai et al., 2017), BioASQ ( http://bioasq. org/ ), RE (Levy et al., 2017), DuoRC (Saha et al., 2018), and DROP (Dua et al., 2019). model (+ 3 .”
Towards VQA Models That Can Read (2019, background): “Textbook QA (TQA) [24] considers the task of answering questions from middle-school textbooks, which often require understanding and reasoning about text and diagrams.”
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering (2022, neutral): “Recently, science problem solving benchmarks [17] have been used to diagnose the multi-hop reasoning ability and interpretability of AI systems.”
IQA: Visual Question Answering in Interactive Environments (2017, background): “These include datasets of natural images [2, 36, 41, 68], synthetic images [2, 4, 24, 27, 28], natural videos [23, 65], synthetic videos [32, 49] and multimodal contexts [29].”
MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension (2019, methodology): “TextbookQA (Kembhavi et al., 2017) Text-bookQA is collected from lessons from middle school Life Science, Earth Science, and Physical Science textbooks.”
Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering (2019, background): “Recent work [2,3,10,11,15,29] proposed to learn models of encoder-decoder structure to tackle the VideoQA problem.”
A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Dataset (2018, neutral): “While it is stated that SQuAD requires logical reasoning, the complexity of reasoning required is far less than that for the AI2 standardized tests dataset (Clark and Etzioni, 2016; Kembhavi et al., 2017).”
Question Answering as Global Reasoning Over Semantic Abstractions (2018, methodology): “Since B I DAF was designed for ﬁll-in-the-blank style questions, we follow the variation used by Kembhavi et al. (2017) to apply it to our multiple-choice setting.”
Scene Text Visual Question Answering (2019, background): “The Textbook Question Answering (TQA) dataset [29] aims at answering multimodal questions given a context of text, diagrams and images, but textual information is provided in computer readable format.”
Few-Shot Question Answering by Pretraining Span Selection (2021, methodology): “…format was popularized by SQuAD (Rajpurkar et al., 2016), and has since been adopted by several datasets in various domains (Trischler et al., 2017; Kembhavi et al., 2017) and languages (Lewis et al., 2020; Clark et al., 2020), with some extensions allowing for unanswerable questions (Levy et al.,…”
An Exploration of Data Augmentation and Sampling Techniques for Domain-Agnostic Question Answering (2019, neutral): “85 TextbookQA (Kembhavi et al., 2017) 56.”
HEAD-QA: A Healthcare Dataset for Complex Reasoning (2019, methodology): “For this reason, our baselines are based on state-of-the-art systems used in open-domain and multi-choice QA (Chen et al., 2017; Kembhavi et al., 2017; Khot et al., 2018; Clark et al., 2018).”
Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling (2024, neutral): “…V3Det (en & zh) [243], DsLMF (en) [272], COCO-ReM (en & zh) [214], TolokaVQA (en) [236] Science AI2D (en) [109], ScienceQA (en) [165], TQA (en) [110], ChemVLM Data (en & zh) [129] ALLaVA (en & zh) [25], Viet-ShareGPT4o (vi) [59], Cambrain-GPT4o (en) [234] , RLAIF-V (en) [282], Laion-GPT4V (en)…”
SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval (2020, methodology): “, 2017) Wikipedia Textbook (Kembhavi et al., 2017) K12 Textbook”