Semantic Pooling for Complex Event Analysis in Untrimmed Videos
By Xiaojun Chang, Yaoliang Yu, Yi Yang, Eric P. Xing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
326 citations recorded by the source corpus.
Pooling plays an important role in generating a discriminative video representation. In this paper, we propose a new semantic pooling approach for challenging event analysis tasks (e.g., event detection, recognition, and recounting) in long untrimmed Internet videos, especially when only a few shots/segments are relevant to the event of interest while many other shots are irrelevant or even misleading. The commonly adopted pooling strategies aggregate the shots indifferently in one way or another, resulting in a great loss of information. Instead, in this work we first define a novel notion of semantic saliency that assesses the relevance of each shot with the event of interest. We then prioritize the shots according to their saliency scores since shots that are semantically more salient are expected to contribute more to the final event analysis. Next, we propose a new isotonic regularizer that is able to exploit the constructed semantic ordering information. The resulting nearly-isotonic support vector machine classifier exhibits higher discriminative power in event analysis tasks. Computationally, we develop an efficient implementation using the proximal gradient algorithm, and we prove new and closed-form proximal steps. We conduct extensive experiments on three real-world video datasets and achieve promising improvements.
Subfields: Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/fc8fb68a7e3b79c37108588671c0e1abf374f501
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
Citations in the corpus
- An overview of Internet of Things (IoT): Architectural aspects, challenges, and protocols (2018, methodology): “Current data mining techniques include shallow and deep learning, 125 supervised, 126 - 129 semi-supervised 130 and unsupervised learning, reinforcement learning, inference based learning, genetic learning, and so on.”
- Revealing Event Saliency in Unconstrained Video Collection (2017, unknown): “ULTIMEDIA event detection [1], [2] aims at ranking a sequence of unconstrained videos according to their likelihood of belonging to a certain event, e.g., " wedding ceremony " .”
- One-Shot Neural Architecture Search: Maximising Diversity to Overcome Catastrophic Forgetting (2020, background): “about a previous task(s) after being trained on a new task [9], [22], [26], [28], [34], [44].”
- Replicating a Trading Strategy by Means of LSTM for Financial Industry Applications (2018, background): “process temporal information regarding video, audio and text [15], [16].”
- A Discriminatively Learned CNN Embedding for Person Reidentification (2016, background): “Recently, the CNN has shown potential for learning state-of-the-art feature embeddings or deep metrics [5, 18, 38, 43, 44, 47, 56].”
- An Adaptive Semisupervised Feature Analysis for Video Semantic Recognition (2018, unknown): “FOR THE explosion of videos along with the rapid development of Internet, video semantic recognition has become a fundamental process that allows the creation of several applications [1], such as semantic video indexing and retrieval, object detection and tracking, video event detection and human ac”