Zero-Shot Activity Recognition with Verb Attribute Induction
By Rowan Zellers, Yejin Choi. 2017
54 citations recorded by the source corpus.
In this paper, we investigate large-scale zero-shot activity recognition by modeling the visual and linguistic attributes of action verbs. For example, the verb “salute” has several properties, such as being a light movement, a social act, and short in duration. We use these attributes as the internal mapping between visual and textual representations to reason about a previously unseen action. In contrast to much prior work that assumes access to gold standard attributes for zero-shot classes and focuses primarily on object attributes, our model uniquely learns to infer action attributes from dictionary definitions and distributed word representations. Experimental results confirm that action attributes inferred from language can provide a predictive signal for zero-shot prediction of previously unseen activities.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yejin Choi
- Mohit Bansal
- Ali Farhadi
- Luke S. Zettlemoyer
- Jianfeng Gao
- Noah A. Smith
- Zhiyuan Liu
- Hannaneh Hajishirzi
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Ali Farhadi
- Yejin Choi
- Kristen Grauman
- Jitendra Malik
- Trevor Darrell
- Li Fei-Fei
- Cordelia Schmid
Citations in the corpus
- SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference (2018, background): “Physical knowledge Prior work has studied learning grounded knowledge about objects and verbs: from knowledge bases (Li et al., 2016), syntax parses (Forbes and Choi, 2017), word embed-dings (Lucy and Gauthier, 2017), and images and dictionary definitions (Zellers and Choi, 2017).”
- Neural Motifs: Scene Graph Parsing with Global Context (2017, neutral): “Much of the context we derive could be interpreted as commonsense priors, which have commonly been extracted using auxiliary means [59, 39, 5, 49, 55].”
- Action Modifiers: Learning From Adverbs in Instructional Videos (2019, background): “While some works learn attributes for actions [28, 43, 58], these detect combinations of specific attributes (e.”
- Few-Shot Charge Prediction with Discriminative Legal Attributes (2018, methodology): “Zero-shot learning has also been used in applications besides object recognition, such as activity recognition (Zellers and Choi, 2017) and event recognition (Wu et al., 2014).”
- Language to Action: Towards Interactive Task Learning with Physical Agents (2018, background): “However, except for a few works that look into physical properties of verbs [Forbes and Choi, 2017; Zellers and Choi, 2017], how verbs and their corresponding actions affect the state of the physical world is largely under-explored.”
- PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World (2021, methodology): “, 2013) and dictionary definitions (Zellers and Choi, 2017).”