ON THE EFFECTIVENESS OF TASK GRANULARITY FOR TRANSFER LEARNING
By Farzaneh Mahdisoltani, Guillaume Berger, Waseem Gharbieh, David J. Fleet, Roland Memisevic. 2018
71 citations recorded by the source corpus.
We describe a DNN for video classification and captioning, trained end-to-end, with shared features, to solve tasks at different levels of granularity, exploring the link between granularity in a source task and the quality of learned features for transfer learning. For solving the new task domain in transfer learning, we freeze the trained encoder and fine-tune a neural net on the target domain. We train on the Something-Something dataset with over 220, 000 videos, and multiple levels of target granularity, including 50 action groups, 174 fine-grained action categories and captions. Classification and captioning with Something-Something are challenging because of the subtle differences between actions, applied to thousands of different object classes, and the diversity of captions penned by crowd actors. Our model performs better than existing classification baselines for SomethingSomething, with impressive fine-grained results. And it yields a strong baseline on the new Something-Something captioning task. Experiments reveal that training with more fine-grained tasks tends to produce better features for transfer learning.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yu Qiao
- Limin Wang
- Ziwei Liu
- Luc Van Gool
- Ping Luo
- Gabriella Csurka
- Rogério Schmidt Feris
- Rameswar Panda
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Yu Qiao
- Andrew Zisserman
- Xiaogang Wang
- Dacheng Tao
- Andrea Vedaldi
- Trevor Darrell
- Jitendra Malik
Citations in the corpus
- Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddings (2019, background): “[20] train for four different tasks, including both coarse and fine grain action recognition.”
- InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation (2023, methodology): “For the referenced public video data, we leverage Kinetics [51], SomethingSomething series [52, 53], UCF101 [54], and so on.”
- AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition (2021, methodology): “, 2017) & V2 (Mahdisoltani et al., 2018), Jester (Materzynska et al.”
- RubiksNet: Learnable 3D-Shift for Efficient Video Action Recognition (2020, unknown): “We leverage the Something-Something-v1 [6], Something-Something-v2 [17], UCF-101 [22], and HMDB [15] datasets to benchmark our approach.”