Sympathy for the Details: Dense Trajectories and Hybrid Classification Architectures for Action Recognition
By César Roberto De Souza, Adrien Gaidon, E. Vig, A. L. Peña. 2016
44 citations recorded by the source corpus.
Action recognition in videos is a challenging task due to the complexity of the spatio-temporal patterns to model and the difficulty to acquire and learn on large quantities of video data. Deep learning, although a breakthrough for image classification and showing promise for videos, has still not clearly superseded action recognition methods using hand-crafted features, even when training on massive datasets. In this paper, we introduce hybrid video classification architectures based on carefully designed unsupervised representations of hand-crafted spatio-temporal features classified by supervised deep networks. As we show in our experiments on five popular benchmarks for action recognition, our hybrid model combines the best of both worlds: it is data efficient (trained on 150 to 10000 short clips) and yet improves significantly on the state of the art, including recent deep models trained on millions of manually labelled images and videos.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Abhinav Kumar Gupta
- Rohit Girdhar
- Kristen Grauman
- Andrew Zisserman
- Li Fei-Fei
- Jitendra Malik
- Trevor Darrell
- Xiaogang Wang
Citations in the corpus
- VideoLSTM convolves, attends and flows for action recognition (2016, neutral): “This implies that our end-to-end prediction is highly complementary to hand-crafted approaches, as previously also observed by others (Feichtenhofer et al., 2016b; Lev et al., 2016; de Souza et al., 2016).”
- Asynchronous Temporal Fields for Action Recognition (2016, background): “With the advent of deep learning, learning representations from data has been extensively studied [14, 15, 44, 57, 52, 53, 24, 7, 61, 55, 40, 3].”
- Learnable pooling with Context Gating for video classification (2017, neutral): “Successful hand-crafted representations [7], [8], [9] are based on local histograms of image and motion gradient orientations extracted along dense trajectories [9], [24].”
- ActionVLAD: Learning Spatio-Temporal Aggregation for Action Classification (2017, background): “CNN-based representations [15, 51, 58, 59, 63] have not yet significantly outper-formed the best hand-engineered descriptors [12, 53].”