ProtoGAN: Towards Few Shot Learning for Action Recognition
By Sai Kumar Dwivedi, Vikram Gupta, Rahul Mitra, Shuaib Ahmed, Arjun Jain. 2019
108 citations recorded by the source corpus.
Few-shot learning (FSL) for action recognition is a challenging task of recognizing novel action categories which are represented by few instances in the training data. In a more generalized FSL setting (G-FSL), both seen as well as novel action categories need to be recognized. Conventional classifiers suffer due to inadequate data in FSL setting and inherent bias towards seen action categories in G-FSL setting. In this paper, we address this problem by proposing a novel ProtoGAN framework which synthesizes additional examples for novel categories by conditioning a conditional generative adversarial network with class prototype vectors. These class prototype vectors are learnt using a Class Prototype Transfer Network (CPTN) from examples of seen categories. Our synthesized examples for a novel class are semantically similar to real examples belonging to that class and is used to train a model exhibiting better generalization towards novel classes. We support our claim by performing extensive experiments on three datasets: UCF101, HMDB51 and Olympic-Sports. To the best of our knowledge, we are the first to report the results for G-FSL and provide a strong benchmark for future research. We also outperform the state-of-the-art method in FSL for all the aforementioned datasets.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Bernt Schiele
- Zeynep Akata
- Trevor Darrell
- Li Fei-Fei
- Yongqin Xian
- Yun Raymond Fu
- Lorenzo Torresani
- Juan Carlos Niebles
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Bernt Schiele
- Trevor Darrell
- Li Fei-Fei
- Dacheng Tao
- Andrew Zisserman
- Cordelia Schmid
- Ming-Hsuan Yang
Citations in the corpus
- Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting (2021, methodology): “We observe that the B /B performance achieves almost 99 points with the previously introduced split by Dwivedi [6] that we report in Table 8.”
- Action Genome: Actions As Compositions of Spatio-Temporal Scene Graphs (2019, methodology): “OSS and TARN propose a measurement of similarity or distance measure between video pairs [7, 39], CMN encodes uses a multi-saliency algorithm to encode videos [90], and ProtoGAN creates a prototype vector for each class [18].”
- Few-Shot Video Classification via Temporal Alignment (2019, methodology): “However, including TARN, previous methods collapse the order of frames at the representation level [7, 13, 24, 25, 28, 48].”
- Human Action Recognition and Prediction: A Survey (2018, background): “ProtoGAN (Dwivedi et al., 2019) learns the class-prototype vectors through a feature aggregator network calledClass Prototype Transfer Network (CPTN), then”
- Generalized Many-Way Few-Shot Video Classification (2020, background): “Recently, a low-shot video classification setup has been proposed [48,7].”