RubiksNet: Learnable 3D-Shift for Efficient Video Action Recognition

By Linxi (Jim) Fan, Shyamal Buch, Guanzhi Wang, Ryan Cao, Yuke Zhu, Juan Carlos Niebles, Li Fei-Fei. 2020

73 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Endocrinology, Diabetes and Metabolism, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Trevor Darrell
2. Li Fei-Fei
3. Kate Saenko
4. Quoc V. Le
5. Luc Van Gool
6. Boqing Gong
7. Abhinav Kumar Gupta
8. Alan L. Yuille

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Trevor Darrell
4. Li Fei-Fei
5. Jitendra Malik
6. Cordelia Schmid
7. Xiaogang Wang
8. Andrew Zisserman

In Endocrinology, Diabetes and Metabolism

1. Aude Oliva
2. Angela Yao
3. Yue Meng
4. Yandong Li
5. Rogério Schmidt Feris
6. Dipika Singhania
7. Adrien Gaidon
8. Rameswar Panda

In Efficiency & Compression

1. Rameswar Panda
2. Shiji Song
3. Gao Huang
4. Kate Saenko
5. Aude Oliva
6. Yulin Wang
7. Rogério Schmidt Feris
8. Le Yang

Citations in the corpus

• Dynamic Neural Networks: A Survey (2021, background): “Adjusting multiple factors of deep models simultaneously has attracted researches in both static [193], [194] and dynamic networks [195], [196], [197], [198].”
• Class-Incremental Learning for Action Recognition in Videos (2021, neutral): “Recent approaches handle this issue in diverse ways, for example, by applying group convolutions [37], learning 3D shift operations [7], decomposing 3D convolution filters [38], searching efficient architectures [8], etc .”
• MoViNets: Mobile Video Networks for Efficient Video Recognition (2021, background): “to improve 2D CNNs’ accuracy using temporal aggregation [40, 17, 70, 43, 16], however their limited inter-frame interactions reduce these models’ abilities to adequately model long-range temporal dependencies like 3D CNNs.”
• Video Transformer Network (2021, background): “, image classification [32, 19, 34], object detection [17, 16, 28], semantic segmentation [25], object instance segmentation [18], face recognition [33, 30] and video action recognition [9, 38, 3, 39, 23, 14, 13, 12].”
• MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge (2022, methodology): “We sample 16 RGB frames from each video uniformly, and apply temporally-consistent random resized crop [13, 26] as data augmentation.”
• Revisiting the “Video” in Video-Language Understanding (2022, methodology): “To ensure learn-ability, we consider two versions of our selector S during training , both operating on the logits g : the first employs a straight-through Gumbel-Softmax estimator [18], the second applies softmax and ensures entropy decreases over time [9].”
• VA-RED2: Video Adaptive Redundancy Reduction (2021, methodology): “(Fan et al., 2020) incorporate a efficient learnable 3D-shift module into a 3D video network.”
• Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities (2022, methodology): “Anticipating actions before they occur is a recently introduced task popularized by EPIC-KITCHENS [5] and Breakfast [22].”
• FrameExit: Conditional Early Exiting for Efficient Video Recognition (2021, methodology): “This can be obtained by decomposing 3D filters into separate 2D spatial and 1D temporal filters [43, 36, 53], extending efficient 2D architectures to their 3D counterparts [42, 28], using shifting operations [31, 9], or exploring neural architecture search for videos [35, 11].”