Efficient Two-Stream Motion and Appearance 3D CNNs for Video Classification
By Ali Diba, Ali Mohammad Pazandeh, Luc Van Gool. 2016
77 citations recorded by the source corpus.
The video and action classification have extremely evolved by deep neural networks specially with two stream CNN using RGB and optical flow as inputs and they present outstanding performance in terms of video analysis. One of the shortcoming of these methods is handling motion information extraction which is done out side of the CNNs and relatively time consuming also on GPUs. So proposing end-to-end methods which are exploring to learn motion representation, like 3D-CNN can achieve faster and accurate performance. We present some novel deep CNNs using 3D architecture to model actions and motion representation in an efficient way to be accurate and also as fast as real-time. Our new networks learn distinctive models to combine deep motion features into appearance model via learning optical flow features inside the network.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Luc Van Gool
- Juergen Gall
- Rainer Stiefelhagen
- Vivek Sharma
- Wanli Ouyang
- Ali Diba
- Alexander G. Hauptmann
- Zhenzhong Lan
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Yi Yang
- Xiaogang Wang
- Dacheng Tao
- Trevor Darrell
- Wanli Ouyang
- Bernt Schiele
- Jitendra Malik
Citations in the corpus
- Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action Classification and Detection (2017, background): “More recently, several CNN based works presented efficient deep models for action recognition [7, 31, 41].”
- Temporal Segment Networks for Action Recognition in Videos (2017, neutral): “Two-Stream 3D ConvNet [68] - 90:2% 246 RGB+EMV [17] - 86.”
- Hidden Two-Stream Convolutional Networks for Action Recognition (2017, methodology): “Supervised optical flow learning methods [18], [23], [55], [56] show that using optical flow from traditional methods (which we call proxy ground truths) can help CNNs to learn to predict motions.”
- Spatio-Temporal Channel Correlation Networks for Action Classification (2018, background): “RGB→Depth [22], RGB→Optical-Flow [22,23], RGB→Sound [24], Near-Infrared→RGB [25]) is effective, and leads to significant improvements in performance.”
- Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition (2017, methodology): “Many approaches learn to capture the motion information directly from input frames using 3D CNN [35, 37, 5, 36, 9, 38].”
- MARS: Motion-Augmented RGB Stream for Action Recognition (2019, unknown): “Other approaches, such as [4, 26] used an off-the-shell optical flow method ([1] in [4], EpicFlow [29] in [26]) as pseudo ground-truth flow.”
- DynamoNet: Dynamic Action and Motion Network (2019, background): “We share the same motivation with these works [8, 11, 35, 36, 46] for learning motion representation in deep learning fashion, but without using optical flow information as the target output.”