BlitzNet: A Real-Time Deep Network for Scene Understanding

By Nikita Dvornik, Konstantin Shmelkov, Julien Mairal, Cordelia Schmid. 2017

211 citations recorded by the source corpus.

Real-time scene understanding has become crucial in many applications such as autonomous driving. In this paper, we propose a deep architecture, called BlitzNet, that jointly performs object detection and semantic segmentation in one forward pass, allowing real-time computations. Besides the computational gain of having a single network to perform several tasks, we show that object detection and semantic segmentation benefit from each other in terms of accuracy. Experimental results for VOC and COCO datasets show state-of-the-art performance for object detection and segmentation among real time systems.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Diversity With Cooperation: Ensemble Methods for Few-Shot Classification (2019, methodology): “Convolutional neural networks [17] have become standard tools in computer vision to model images, leading to out-standing results in many tasks such as classiﬁcation [16], object detection [8, 19, 25], or semantic segmentation [8, 20, 26].”
YOLACT: Real-Time Instance Segmentation (2019, methodology): “Real-time Instance Segmentation While real-time object detection [30, 34, 35, 36], and semantic segmentation [2, 41, 33, 11, 47] methods exist, few works have focused on real-time instance segmentation.”
Modeling Visual Context is Key to Augmenting Object Detection Datasets (2018, methodology): “The detector is trained by following [23], with the ADAM optimizer [38] starting from learning rate 10 − 4 and decreasing it later during training by a factor 10 (see Sections 4.3 and 4.4 for the number of epochs used in each experiment).”
Multi-Task Collaborative Network for Joint Referring Expression Comprehension and Segmentation (2020, background): “It is a natural thought to jointly learn REC and RES to reinforce each other, as similar to the classic endeavors in joint object detection and segmentation [9, 10, 7].”
YOLACT++ Better Real-Time Instance Segmentation (2019, methodology): “Real-time Instance Segmentation While real-time object detection [1], [6], [7], [26], and semantic segmentation [27], [28], [29], [30], [31] methods exist, few works have focused on real-time instance segmentation.”
AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning (2019, background): “Hard-parameter sharing is the most common MTL approach that usually relies on hand-designed network architectures composed of hidden layers that are shared across all tasks and specialized branches that learn task-speciﬁc features [19, 26, 39, 5, 22, 11].”
FSSD: Feature Fusion Single Shot Multibox Detector (2017, unknown): “BlitzNet512 [5] 07++12+S+COCO ResNet-50 ! 83.”
Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotations (2018, methodology): “BlitzNet [11] 25405D + 795SD 795SD 1 1 34.”
Receptive Field Block Net for Accurate and Fast Object Detection (2017, methodology): “BlitzNet [5] is a recent deep system for object detection and segmentation, using both bounding boxes and segmentation labels in training.”
Few-Shot Semantic Segmentation with Prototype Learning (2018, methodology): “These techniques usually modify the original images, such as adding noise to the raw images, making geometric transformation and using part of the images [10, 13, 14, 32, 34].”
Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification From the Bottom Up (2019, unknown): “tracking [42, 33, 38], and scene understanding [8, 31].”
Representation Similarity Analysis for Efficient Task Taxonomy & Transfer Learning (2019, background): “For instance, in multi-task learning [17, 13, 7, 21, 8] RSA could be used for deciding different branching out locations for different tasks, depending on their similarity with the representations at different depth of the shared root.”
Retina U-Net: Embarrassingly Simple Exploitation of Segmentation Supervision for Medical Object Detection (2018, unknown): “On the other hand, one-stage detectors have been proposed, where category-speciﬁc classiﬁcation is performed immediately on the coarse representations [21, 18, 16].”
Attentive Single-Tasking of Multiple Tasks (2019, methodology): “Successful pairs include detection and classiﬁcation [19, 48], detection and segmentation [22, 12], or monocular depth and segmentation [13, 64].”
Triply Supervised Decoder Networks for Joint Detection and Segmentation (2018, methodology): “Namely, the proposed TripleNet outperforms BlitzNet [8] by 1.”
Multi-Task Learning for Dense Prediction Tasks: A Survey (2020, background): “depth and flow estimation [80], detection and classification [81], [82], detection and segmentation [2], [83].”
MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning (2020, background): “n, or act as a regularizer for one another. Evidence for the former has been provided in the literature for certain pairs of tasks, e.g. detection and classication [9,34], detection and segmentation [6,12], segmentation and depth estimation [7,47], while for the latter recent eorts point to that dir”