Video Question Answering via Gradually Refined Attention over Appearance and Motion

By Dejing Xu, Zhou Zhao, Jun Xiao, Fei Wu, Hanwang Zhang, Xiangnan He, Yueting Zhuang. 2017

817 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Question Answering & Reading Comprehension

Citations in the corpus

Flamingo: a Visual Language Model for Few-Shot Learning (2022, background): “[130] Dejing Xu, Zhou Zhao, Jun Xiao, Fei Wu, Hanwang Zhang, Xiangnan He, and Yueting Zhuang.”
STAR: A Benchmark for Situated Reasoning in Real-World Videos (2024, background): “The existing visual/video question answering benchmarks [13, 50, 24, 40, 44] adopted images [1, 50, 11]/videos [40, 44, 19, 28, 18, 9, 45] and types of visual comprehension questions.”
All in One: Exploring Unified Video-Language Pre-Training (2022, background): “The pre-train and then fine-tune scheme, has become a standard paradigm to learn transferable video-language representations for a wide range of downstream video-text tasks [6, 15, 52, 52, 53, 61].”
MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound (2022, methodology): “We evaluate on four diverse benchmarks: Situated Reasoning (STAR) [119], EPIC-Kitchens [26], LSMDC-FiB [95], and MSR-VTT QA [120].”
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling (2021, methodology): “Choice TGIF-Action [6] 5 TGIF-Transition [6] 5 MSRVTT-MC [7] 5 LSMDC-MC [3] 5”
MERLOT: Multimodal Neural Script Knowledge Models (2021, methodology): “We report results on 12 video reasoning tasks: TVQA [64], TVQA(+) [65], VLEP [66], MSRVTT-QA [117], MSRVTT-Multichoice [121], LSMDC-Multichoice, LSMDC ﬁll-in-the-blank QA [110, 92], ActivityNetQA [122, 45], TGIFQA [52], and DramaQA [23].”
NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions (2021, methodology): “MSVD-QA [51] MSVD descriptive QA Auto 1,970 50,505 10 OC MSRVTT-QA [51] MSRVTT descriptive QA Auto 10,000 243,690 15 OC TGIF-QA [17, 18] TGIF spatio-temporal reasoning Auto 71,741 165,165 3 MC&OC TVQA [25] TV Show subtitle&concept comprehension Man 21,793 152,545 76 MC ActivityNet-QA [56] ActivityNe”
Just Ask: Learning to Answer Questions from Millions of Narrated Videos (2020, neutral): “Initial experiments showed that existing benchmarks for open-ended VideoQA [87, 94] contain a language bias [29], i.e., their questions can often be answered without looking at the video.”
GIT: A Generative Image-to-text Transformer for Vision and Language (2022, methodology): “Although our model is not dedicated for video tasks, our model achieve new SOTA on MSRVD, MSRVTT, and VATEX for captioning and on MSVD-QA and TGIF-Frame for QA.”
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation (2021, background): “5 There are other VidL datasets on single-channel videos that are not included in VALUE, due to the difficulties in collecting hidden test set [11, 70, 8], unnatural annotations [69], or the lack of subtitle channel in GIF videos [25].”
Divide and Conquer: Question-Guided Spatio-Temporal Contextual Attention for Video Question Answering (2020, methodology): “To further evaluate the effectiveness of the proposed model, we have also tested our QueST on other Video QA datasets: MSRVTT-QA (Xu et al. 2017) and MSVD-QA (Xu et al.”
Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learning (2020, methodology): “…complex and challenging artificial intelligence tasks such as Visual Question Answering (Antol et al. 2015; Jang et al. 2017; Gao et al. 2018; Xu et al. 2017; Fan et al. 2019) and Text-to-Video Retrieval (Liu et al. 2019; Mithun et al. 2018; Yu, Kim, and Kim 2018; Miech, Laptev, and Sivic…”
Look Before you Speak: Visually Contextualized Utterances (2020, methodology): “Concretely, we make the following four contributions: (i) We formulate a future utterance prediction (FUP) task which uses both dialogue and vision; (ii) We re-purpose freely available online instructional video datasets to create training and testing benchmarks for this task (HowToFUP); (iii) We pr”
Location-Aware Graph Convolutional Networks for Video Question Answering (2020, methodology): “The main contributions of the proposed method are as follows: (1) we propose to explore actions for video QA task through learning interaction between detected objects such that irrelevant background content can be explicitly excluded; (2) we propose to model the relationships between objects throug”
ShareGPT4Video: Improving Video Understanding and Generation with Better Captions (2024, background): “…[44] curates approximately 15,000 QA pairs spanning 10 evaluation dimensions from 13 existing data sources, such as MSVD-QA [62], MSRVTT-QA [62], Activitynet-QA [67], etc. MVBench [30] is designed to challenge LVLMs with video tasks that cannot be effectively resolved by single-frame…”