Attentional ShapeContextNet for Point Cloud Recognition

By Saining Xie, Sainan Liu, Zeyu Chen, Zhuowen Tu. 2018

344 citations recorded by the source corpus.

We tackle the problem of point cloud recognition. Unlike previous approaches where a point cloud is either converted into a volume/image or represented independently in a permutation-invariant set, we develop a new representation by adopting the concept of shape context as the building block in our network design. The resulting model, called ShapeContextNet, consists of a hierarchy with modules not relying on a fixed grid while still enjoying properties similar to those in convolutional neural networks - being able to capture and propagate the object part information. In addition, we find inspiration from self-attention based models to include a simple yet effective contextual modeling mechanism - making the contextual region selection, the feature aggregation, and the feature transformation process fully automatic. ShapeContextNet is an end-to-end model that can be applied to the general point cloud classification and segmentation problems. We observe competitive results on a number of benchmark datasets.

Subfields: Computational Mechanics, Environmental Engineering, Geology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Environmental Engineering

In Geology

Citations in the corpus

SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks (2020, background): “Motivated by a successes across a wide range of tasks in deep learning such as language modeling [31], image recognition [18], graph-based problems [32], and relational reasoning [30, 9], a recent stream of work has applied forms of self-attention algorithms to point cloud data [44, 42, 16].”
Deep Hough Voting for 3D Object Detection in Point Clouds (2019, background): “First, given a collection of images with annotated object bounding boxes, a codebook is constructed with stored mappings between image patches (or their features) and their offsets to the corresponding object centers.”
GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping (2020, methodology): “After that, many methods [34, 38, 3, 23, 12, 43, 39, 40, 20, 19, 49, 47, 44, 46, 15] are proposed to perform point cloud classification and segmentation.”
Visual attention network (2022, background): “Due to its effectiveness of capturing the long range dependence and adaptability, it is playing an increasingly important role in computer vision [83,17,61,96,98,90].”
Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review (2020, neutral): “Those approaches [141]–[143] have demonstrated significant”
PointASNL: Robust Point Clouds Processing Using Nonlocal Neural Networks With Adaptive Sampling (2020, background): “A-SCN [46] uses a global attention mechanism to aggregate global features but lacks the support of local information, which does not achieve good results.”
RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds (2019, background): “These modules can be generally categorized as: 1) neighbouring feature pooling [44, 32, 21, 70, 69], 2) graph message passing [57, 48, 55, 56, 5, 22, 34], 3) kernel-based convolution [49, 20, 60, 29, 23, 24, 54, 38], and 4) attentionbased aggregation [61, 68, 66, 42].”
Rotation Invariant Convolutions for 3D Point Clouds Deep Learning (2019, methodology): “Recent methods explores convolution kernels that exploit geometric features [30], add edges on top of points [35], parameterize convolution using polynomials [39], and leverage shape context [38].”
Transductive Zero-Shot Learning for 3D Point Cloud Classification (2019, methodology): “Any network that learns a feature space from 3D point sets and is invariant to permutations of points in the point cloud can be used in our method as the point cloud network [30, 31, 55, 24, 61, 54, 60].”
Revealing Occlusions with 4D Neural Fields (2022, background): “More recently, researchers have begun to apply transformer attention mechanisms that were ﬁrst found to be valuable in the language domain [58] to encode point clouds [27,28,64].”
Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts (2020, methodology): “There are many ways to achieve the goal, and here we take inspiration from the classic ShapeContext local descriptor [4, 35, 67] for shape matching.”
Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation (2019, methodology): “Similarly, employing FiLM modulation has been shown effective on a variety of tasks such as image synthesis [19, 36, 17, 1], visual question answering [38, 37], style transfer [7], recognition [16, 60], reading comprehension [5], etc.”
Attention mechanisms in computer vision: A survey (2021, methodology): “[97, 46, 139, 70, 128, 36, 11, 156] , [137, 143, 63, 161, 159, 6, 177, 80] , [67, 180, 10, 23, 24, 163, 155, 151] , [44, 49, 106, 64, 168, 18, 11, 36] , [48, 16, 169, 153, 174, 127, 53, 92] , [89, 21, 5, 134, 65, 171, 2, 109] , [45, 82, 157, 91, 100, 129, 146, 131] , [138, 85, 175, 76, 166, 133, 167”
DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing (2019, background): “nt: point coordinates, nor: normal, “-”: unknown). method input #points M40 M10 Pointwise-CNN [12] pnt 1k 86.1 - Deep Sets [60] pnt 1k 87.1 - ECC [40] pnt 1k 87.4 90.8 PointNet [32] pnt 1k 89.2 - SCN [55] pnt 1k 90.0 - Kd-Net(depth=10) [21] pnt 1k 90.6 93.3 PointNet++ [34] pnt 1k 90.7 - MC-Conv [11]”
Modeling Point Clouds With Self-Attention and Gumbel Subset Sampling (2019, background): “, graph neural networks on point sets [46, 34, 23, 22], or 3) hybrid [55, 21, 36].”
Embodied Question Answering in Photorealistic Environments With Point Cloud Perception (2019, background): “To overcome the ordering issue, some utilize symmetric functions, PointNet(++) [31, 32], and ASCN [33].”
Relation-Shape Convolutional Neural Network for Point Cloud Analysis (2019, neutral): “k 80.4 83.7 83.4 78.7 82.5 74.9 89.6 73.0 91.5 85.9 80.8 95.3 65.2 93.0 81.2 57.9 72.8 80.6 RS-Net [12] - 81.4 84.9 82.7 86.4 84.1 78.2 90.4 69.3 91.4 87.0 83.5 95.4 66.0 92.6 81.8 56.1 75.8 82.2 SCN [47] 1k 81.8 84.6 83.8 80.8 83.5 79.3 90.5 69.8 91.7 86.5 82.9 96.0 69.2 93.8 82.5 62.9 74.4 80.8 PC”
Point Transformer (2020, methodology): “We compare our results with approaches that operate directly on 3D point sets [7], [13], [14], attentionbased approaches [17], [20], [35] and methods that use point cloud convolutions [29]–[31], [39].”