RareAct: A video dataset of unusual interactions
By Antoine Miech, Jean-Baptiste Alayrac, Ivan Laptev, Josef Sivic, Andrew Zisserman. 2020
30 citations recorded by the source corpus.
This paper introduces a manually annotated video dataset of unusual actions, namely RareAct, including actions such as "blend phone", "cut keyboard" and "microwave shoes". RareAct aims at evaluating the zero-shot and few-shot compositionality of action recognition models for unlikely compositions of common action verbs and object nouns. It contains 122 different actions which were obtained by combining verbs and nouns rarely co-occurring together in the large-scale textual corpus from HowTo100M, but that frequently appear separately. We provide benchmarks using a state-of-the-art HowTo100M pretrained video and text model and show that zero-shot and few-shot compositionality of actions remains a challenging and unsolved task.
Subfields: Computer Vision and Pattern Recognition, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Andrew Zisserman
- Trevor Darrell
- Dacheng Tao
- Ming-Hsuan Yang
- Jitendra Malik
- Andrea Vedaldi
- Cordelia Schmid
In Evaluation & Benchmarks
- Andrew Zisserman
- Karen Simonyan
- Jeff Donahue
- Jean-Baptiste Alayrac
- Shijian Lu
- Sheng Jin
- Jacob Menick
- Pauline Luc
Citations in the corpus
- Learning Transferable Visual Models From Natural Language Supervision (2021, methodology): “Several works have explored using dense spoken natural language supervision for videos (Miech et al., 2019; 2020b).”
- Flamingo: a Visual Language Model for Few-Shot Learning (2022, methodology): “Note: We omit RareAct, our 16th benchmark, as it is a zero-shot benchmark with no available fine-tuned results to compare to.”
- Human Action Recognition From Various Data Modalities: A Review (2020, unknown): “[407] 2019 WiFi CSI 6 1 1,394 NTU RGB+D 120 [151] 2019 RGB,S,D,IR 120 106 114,480 155 IKEA ASM [408] 2020 RGB,S,D 33 48 16,764 3 EPIC-KITCHENS-100 [409] 2020 RGB,Au,Ac - 45 89,979 Egocentric ETRI-Activity3D [410] 2020 RGB,S,D 55 100 112,620 RareAct [411] 2020 RGB 122 - 905 BABEL [412] 2021 Mocap 252”
- Vision-Language Models for Vision Tasks: A Survey (2023, background): “Image classification [5], [6] and action recognition [28], [29] have been widely adopted in such evaluations, where video clips are often sub-sampled for efficient recognition in action recognition tasks [10].”