Dense Regression Network for Video Grounding

By Runhao Zeng, Haoming Xu, Wenbing Huang, Peihao Chen, Mingkui Tan, Chuang Gan. Computer Vision and Pattern Recognition, 2020

339 citations recorded by the source corpus.

We address the problem of video grounding from natural language queries. The key challenge in this task is that one training video might only contain a few annotated starting/ending frames that can be used as positive examples for model training. Most conventional approaches directly train a binary classifier using such imbalance data, thus achieving inferior results. The key idea of this paper is to use the distances between the frame within the ground truth and the starting (ending) frame as dense supervisions to improve the video grounding accuracy. Specifically, we design a novel dense regression network (DRN) to regress the distances from each frame to the starting (ending) frame of the video segment described by the query. We also propose a simple but effective IoU regression head module to explicitly consider the localization quality of the grounding results (i.e., the IoU between the predicted location and the ground truth). Experimental results show that our approach significantly outperforms state-of-the-arts on three datasets (i.e., Charades-STA, ActivityNet-Captions, and TACoS).

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/355403d7ce4b625307fd3ebb2beea269ecc15213

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Mingkui Tan
2. Chuang Gan
3. Qi Wu
4. Luc Van Gool
5. Ming-Hsuan Yang
6. Yin Li
7. Peilin Zhao
8. Yifan Zhang

Citations in the corpus

• Language Models Can See: Plugging Visual Controls in Text Generation (2022, methodology): “Beyond describing the whole image scene, dense captioning methods [30, 5, 35, 88, 89] aim to describe the visual objects in a sub-region of the input image.”
• RSPNet: Relative Speed Perception for Unsupervised Video Representation Learning (2020, background): “Video analysis [24] has been a prominent research topic in computer vision due to its vast potential applications, such as action recognition [32, 8], event detection [14], action localization [4, 49, 50], audio-visual scene analysis [16, 5, 12], etc .”
• Towards Accurate Text-based Image Captioning with Content Diversity Exploration (2021, background): “[22] are the first to propose the dense captioning task to describe the visual objects in a sub-region [50].”
• ActionFormer: Localizing Moments of Actions with Transformers (2022, background): “Anchor-free temporal localization [81] has been explored in video grounding.”
• Efficient Test-Time Model Adaptation without Forgetting (2022, background): “, 2016), video recognition (Wang et al., 2018; Zeng et al., 2020; 2021; Chen et al., 2021), and many other areas (Choi et al.”
• Embracing Uncertainty: Decoupling and De-bias for Robust Temporal Grounding (2021, methodology): “First of all, we compare our model DeNet with other state-of-the-art methods using standard metrics on two datasets, which contains CTRL [6], SMRL [38], MAC [8], MLVI [39], CBP [36], SAP [4], MAN [46], 2D-TAN [47], EXCL [9], TMLGA [9], DRN [45], SCDM [43], LGI [24] and ABLR [44].”