PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition
By Haoxuan You, Yifan Feng, Rongrong Ji, Yue Gao. 2018
191 citations recorded by the source corpus.
3D object recognition has attracted wide research attention in the field of multimedia and computer vision. With the recent proliferation of deep learning, various deep models with different representations have achieved the state-of-the-art performance. Among them, point cloud and multi-view based 3D shape representations are promising recently, and their corresponding deep models have shown significant performance on 3D shape recognition. However, there is little effort concentrating point cloud data and multi-view data for 3D shape representation, which is, in our consideration, beneficial and compensated to each other. In this paper, we propose the Point-View Network (PVNet), the first framework integrating both the point cloud and the multi-view data towards joint 3D shape recognition. More specifically, an embedding attention fusion scheme is proposed that could employ high-level features from the multi-view data to model the intrinsic correlation and discriminability of different structure features from the point cloud data. In particular, the discriminative descriptions are quantified and leveraged as the soft attention mask to further refine the structure feature of the 3D shape. We have evaluated the proposed method on the ModelNet40 dataset for 3D shape classification and retrieval tasks. Experimental results and comparisons with state-of-the-art methods demonstrate that our framework can achieve superior performance.
Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
In Computer Vision and Pattern Recognition
In Geology
Citations in the corpus
- Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data (2019, methodology): “There are also methods jointly learning features from point clouds and multi-view projections [47].”
- Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework (2022, background): “…cloud data structure is irregular and unordered, some works consider projecting the original point clouds to intermediate voxels (Maturana & Scherer, 2015; Shi et al., 2020) or images (You et al., 2018; Li et al., 2020), translating the challenging 3D task into a well-explored 2D image problem.”
- MeshNet: Mesh Neural Network for 3D Shape Representation (2018, background): “Point-view network (PVNet) (You et al. 2018) proposes the embedding attention fusion to exploit both point cloud data and multi-view data.”
- MVTN: Multi-View Transformation Network for 3D Shape Recognition (2020, unknown): “Voxels 40.9 - 3D ShapeNets [34] Voxels 49.2 - PVNet[36] Points 89.5 - MVCNN [28] 12 Views 80.2 73.5 GIFT [1] 20 Views - 64.0 MVFusionNet [12] 12 Views - 62.2 ReVGG [27] 20 Views - 74.9 RotNet [14] 20 Views - 77.2 ViewGCN [32] 20 Views - 78.4 MLVCNN [13] 24 Views 92.2 - MVTN (ours) 12 Views 92.9 (92.”
- Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review (2020, background): “The well-developed 2D DL architectures can better exploit the local and global information from projected 2D view images [91].”