Memory Enhanced Global-Local Aggregation for Video Object Detection

By Yihong Chen, Yue Cao, Han Hu, Liwei Wang. Computer Vision and Pattern Recognition, 2020

313 citations recorded by the source corpus.

How do humans recognize an object in a piece of video? Due to the deteriorated quality of single frame, it may be hard for people to identify an occluded object in this frame by just utilizing information within one image. We argue that there are two important cues for humans to recognize objects in videos: the global semantic information and the local localization information. Recently, plenty of methods adopt the self-attention mechanisms to enhance the features in key frame with either global semantic information or local localization information. In this paper we introduce memory enhanced global-local aggregation (MEGA) network, which is among the first trials that takes full consideration of both global and local information. Furthermore, empowered by a novel and carefully-designed Long Range Memory (LRM) module, our proposed MEGA could enable the key frame to get access to much more content than any previous methods. Enhanced by these two sources of information, our method achieves state-of-the-art performance on ImageNet VID dataset. Code is available at https://github.com/Scalsol/mega.pytorch.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3d5dd5c128afa0f11a7cbdf2c3f879c82fca4351

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Luc Van Gool
3. Yunhe Wang
4. Kai Han
5. Xiaogang Wang
6. Jiashi Feng
7. Zheng Zhang
8. Alan L. Yuille

Citations in the corpus

• Disentangled Non-Local Neural Networks (2020, background): “Since then, such modeling methods have demonstrated great effectiveness in many vision tasks, such as image classification [22], object detection [21,15], semantic segmentation [39], video object detection [10,36,16,6] and tracking [37], and action recognition [34].”
• A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit (2021, background): “Many works in the literature use the mAP as a metric to evaluate the performance of object detection models in video sequences [92–98].”
• A Survey on Vision Transformer (2020, background): “introduces the memory enhanced global-local aggregation (MEGA) [19] to capture more content.”
• BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection (2022, background): “The latter reutilizes the features from the previous frame based on some special architectures like LSTM [14] for feature distillation [21, 20, 25], attention mechanism [37] for feature querying [5, 8, 9], and optical flow [10] for feature alignment [52, 51].”