CERN: Confidence-Energy Recurrent Network for Group Activity Recognition
By Tianmin Shu, Sinisa Todorovic, Song-Chun Zhu. 2017
162 citations recorded by the source corpus.
This work is about recognizing human activities occurring in videos at distinct semantic levels, including individual actions, interactions, and group activities. The recognition is realized using a two-level hierarchy of Long Short-Term Memory (LSTM) networks, forming a feed-forward deep architecture, which can be trained end-to-end. In comparison with existing architectures of LSTMs, we make two key contributions giving the name to our approach as Confidence-Energy Recurrent Network – CERN. First, instead of using the common softmax layer for prediction, we specify a novel energy layer (EL) for estimating the energy of our predictions. Second, rather than finding the common minimum-energy class assignment, which may be numerically unstable under uncertainty, we specify that the EL additionally computes the p-values of the solutions, and in this way estimates the most confident energy minimum. The evaluation on the Collective Activity and Volleyball datasets demonstrates: (i) advantages of our two contributions relative to the common softmax and energy-minimization formulations and (ii) a superior performance relative to the state-of-the-art approaches.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Li Fei-Fei
- Silvio Savarese
- Juan Carlos Niebles
- Lu Jiang
- Trevor Darrell
- Justin Johnson
- Alexandre Alahi
- Alexander G. Hauptmann
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Li Fei-Fei
- Trevor Darrell
- Dacheng Tao
- Xiaogang Wang
- Ming-Hsuan Yang
- Jitendra Malik
- Yi Yang
Citations in the corpus
- Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates (2017, neutral): “[25], [26], [27], [28], [29], [30], and RGB-based activity recognition [31], [32], [33], [34], [35], [36].”
- Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks (2018, background): “Recurrent Neural Networks are a rich class of dynamic models which extend feedforward networks for sequence generation in diverse domains like speech recognition [7, 8, 15], machine translation [8] and image captioning [20, 43, 45, 39].”
- STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction (2019, background): “They have achieved great success in many sequence prediction tasks, e.g., speech recognition [7, 11], machine translation [4, 6, 30] and image cap-tioning [8, 26, 37].”
- Spatio-Temporal Dynamic Inference Network for Group Activity Recognition (2021, unknown): “Our models generally cost less computational overhead and perform better than RNN-based models like [45, 32, 4, 28, 35], due to a better representation from jointly modeling of spatial-temporal interaction.”
- Peeking Into the Future: Predicting Future Person Activities and Locations in Videos (2019, background): “There are previous works that take into account multiple cues in videos for tracking [11, 25] and group activity recognition [5, 28, 27].”