Deep MANTA: A Coarse-to-Fine Many-Task Network for Joint 2D and 3D Vehicle Analysis from Monocular Image

By Florian Chabot, Mohamed Chaouch, Jaonary Rabarisoa, Céline Teulière, Thierry Chateau. 2017

466 citations recorded by the source corpus.

In this paper, we present a novel approach, called Deep MANTA (Deep Many-Tasks), for many-task vehicle analysis from a given image. A robust convolutional network is introduced for simultaneous vehicle detection, part localization, visibility characterization and 3D dimension estimation. Its architecture is based on a new coarse-to-fine object proposal that boosts the vehicle detection. Moreover, the Deep MANTA network is able to localize vehicle parts even if these parts are not visible. In the inference, the networks outputs are used by a real time robust pose estimation algorithm for fine orientation estimation and 3D vehicle localization. We show in experiments that our method outperforms monocular state-of-the-art approaches on vehicle detection, orientation and 3D location tasks on the very challenging KITTI benchmark.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Objects as Points (2019, methodology): “Deep Manta [6] uses a coarse-to-fine Faster-RCNN [46] trained on many tasks.”
A General Pipeline for 3D Detection of Vehicles (2018, methodology): “The AP is calculated based on the IoU between the output boxes and the ground truth boxes, while in [9] and [7], the distance between two boxes are used.”
SM-NAS: Structural-to-Modular Neural Architecture Search for Object Detection (2019, unknown): “This task widely benefits autonomous driving [6],”
Joint 3D Proposal Generation and Object Detection from View Aggregation (2017, unknown): “Deep MANTA [14] proposes a many-task vehicle analysis approach from monocular images that”
GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving (2019, methodology): “Among the previous works, [24, 19] use val1, and [2, 3] use val2, and [1, 27] use them both.”
RT3D: Real-Time 3-D Vehicle Detection in LiDAR Point Cloud for Autonomous Driving (2018, background): “[7] proposed a multi-task neural network to create coarse-to-ﬁne object proposals and to estimate vehicle orientation and location through 2D/3D point matching.”
Disentangling Monocular 3D Object Detection (2019, methodology): “Deep-MANTA [3] uses 3D CAD models and annotated 3D parts in a coarseto-fine localization process.”
YOLO3D: End-to-end real-time 3D Oriented Object Bounding Box Detection from LiDAR Point Cloud (2018, background): “[9] and [10] can produce highly accurate 3D bounding boxes using only camera images.”
Geometry Uncertainty Projection Network for Monocular 3D Object Detection (2021, methodology): “DeepMANTA [7] introduced the 3D CAD model to learn shape-based knowledge and guided to better dimension prediction results.”
SP-NAS: Serial-to-Parallel Backbone Search for Object Detection (2020, background): “Object detection is one of the core computer vision tasks to localize and classify multiple objects in an image and has been widely used in real-world applications [29, 28, 7, 2].”
Orthographic Feature Transform for Monocular 3D Object Detection (2018, methodology): “Many approaches start from 2D bounding boxes extracted using standard detectors described above, upon which they either directly regress 3D pose parameters for each region [14, 26, 24, 23] or fit 3D templates to the image [2, 35, 36, 38].”
Delving into Localization Errors for Monocular 3D Object Detection (2021, methodology): “To better estimate the 3D bounding boxes, many methods are proposed for effectively using additional data [38, 26, 5, 39, 4, 12, 16, 13, 37, 25, 24, 3].”
Multi-level Fusion Based 3D Object Detection from Monocular Images (2018, methodology): “Another recently introduced method for monocular 3D object detection is called Deep MANTA [1].”
DepthCN: Vehicle detection using 3D-LIDAR and ConvNet (2017, unknown): “[19] introduced Deep MANTA, a framework for 2D and 3D vehicle detection in monocular images.”
RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving (2020, background): “…Depth Shape/CAD Segmentation Mono3D [7] 3DOP [8], stereoRCNN [22] MF3D [42], Peseudo-LiDAR [39], MonoPSR [17] AM3D[27] Mono3D++ [14],Deep-MENTA [6], 3DVP [40] Deep3DBox [28],GS3D [20],MonoGRNet [33], FQNet[25], M3D-RPN[4] Shift-RCNN [30] Ours(RTM3D) Image-only in Monocular 3D Object Detection.”
Argoverse: 3D Tracking and Forecasting With Rich Maps (2019, methodology): “[8] Florian Chabot, Mohamed Chaouch, Jaonary Rabarisoa, Celine Teulière, and Thierry Chateau.”
MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships (2020, methodology): “Deep MANTA [4] predicts 3D object information utilizing key points and 3D CAD models.”
Categorical Depth Distribution Network for Monocular 3D Object Detection (2021, background): “Shape estimates can be supervised from labeled vertices of 3D CAD models [5, 24], from LiDAR scans [22], or directly from input data in a self-supervised manner [2].”