Group-Free 3D Object Detection via Transformers

By Ze Liu, Zheng Zhang, Yue Cao, Han Hu, Xin Tong. IEEE International Conference on Computer Vision, 2021

387 citations recorded by the source corpus.

Recently, directly detecting 3D objects from 3D point clouds has received increasing attention. To extract object representation from an irregular point cloud, existing methods usually take a point grouping step to assign the points to an object candidate so that a PointNet-like network could be used to derive object features from the grouped points. However, the inaccurate point assignments caused by the hand-crafted grouping scheme decrease the performance of 3D object detection.In this paper, we present a simple yet effective method for directly detecting 3D objects from the 3D point cloud. Instead of grouping local points to each object candidate, our method computes the feature of an object from all the points in the point cloud with the help of an attention mechanism in the Transformers [42], where the contribution of each point is automatically learned in the network training. With an improved attention stacking scheme, our method fuses object features in different stages and generates more accurate object detection results. With few bells and whistles, the proposed method achieves state-of-the-art 3D object detection performance on two widely used benchmarks, Scan-Net V2 and SUN RGB-D. The code and models are publicly available at https://github.com/zeliu98/Group-Free-3D

View Paper: https://www.semanticscholar.org/paper/c2cf5e5f21cca7045be719afce6bf66cc2934370

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Renrui Zhang
2. Xiaogang Wang
3. Wanli Ouyang
4. Luc Van Gool
5. Peng Gao
6. Hongsheng Li
7. Yi Yang
8. Dacheng Tao

Citations in the corpus

• Mask3D: Mask Transformer for 3D Semantic Instance Segmentation (2022, methodology): “domain though and recent Transformer-based methods focus on either 3D object detection [34, 37, 39] or 3D semantic segmentation [29, 40, 63].”
• ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding (2022, background): “3D visual understanding research [6, 12, 16, 17, 24, 25] is drawing significant attention in recent years due to the increasing demand of real-world applications such as augmented/virtual reality [1, 27, 31, 46], autonomous driving [23, 57] and robotics [2, 48].”
• TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers (2022, background): “In this way, our initial object queries will locate at or close to the potential object centers, eliminating the need of multiple decoder layers [20, 23, 49] to refine the locations.”
• ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data (2021, background): “A variety of methods are addressing different parts of the challenge, like depth estimation, 3D reconstruction, instance segmentation [1, 2, 3], object detection [4, 5, 6, 7] and more.”
• A Survey of Visual Transformers (2021, background): “Group-Free 3D DETR [118] applies a more specified and stronger structure than [117].”
• DETRs with Hybrid Matching (2022, background): “The concurrent GroupFree3D [39] also showed the advantages, i.”
• Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework (2022, methodology): “As an example, until now, most 3D point cloud applications are still based on the simple PointNet ( and PointNet++) or the voxel-based methods (Liu et al., 2021; Li et al., 2021b; Zhang et al., 2021).”
• Back to Reality: Weakly-supervised 3D Object Detection with Shape-guided Label Enhancement (2022, methodology): “Different from previous works [20,25], in our setting we need to detect small objects, such as bottle, cup and keyboard.”
• Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following (2023, background): “In these years, 3D vision has gained significant attention and development, driven by the rising popularity of autonomous driving [11, 69, 71], navigation [72, 76, 98], 3D scene understanding [2, 43, 46, 74], and robotics [31, 67].”