FeaStNet: Feature-Steered Graph Convolutions for 3D Shape Analysis

By Nitika Verma, Edmond Boyer, Jakob Verbeek. 2017

363 citations recorded by the source corpus.

Convolutional neural networks (CNNs) have massively impacted visual recognition in 2D images, and are now ubiquitous in state-of-the-art approaches. CNNs do not easily extend, however, to data that are not represented by regular grids, such as 3D shape meshes or other graph-structured data, to which traditional local convolution operators do not directly apply. To address this problem, we propose a novel graph-convolution operator to establish correspondences between filter weights and graph neighborhoods with arbitrary connectivity. The key novelty of our approach is that these correspondences are dynamically computed from features learned by the network, rather than relying on predefined static coordinates over the graph as in previous work. We obtain excellent experimental results that significantly improve over previous state-of-the-art shape correspondence results. This shows that our approach can learn effective shape representations from raw input coordinates, without relying on shape descriptors.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

An End-to-End Transformer Model for 3D Object Detection (2021, background): “Convolutions can also be applied by soft-assignment matrices [69] or specific ordering [28].”
ABC: A Big CAD Model Dataset for Geometric Deep Learning (2018, background): “In a similar fashion, FeaStNet [54] proposes a convolution operator for general graphs, which applies filter kernels in a data-driven manner to the local irregular neighborhoods.”
3D Hand Shape and Pose Estimation From a Single RGB Image (2019, methodology): “Graph CNNs have been successfully applied in modeling graph structured data [56, 59, 55].”
PointConv: Deep Convolutional Networks on 3D Point Clouds (2018, background): “Some latest work [30, 26, 28, 35, 37, 13, 9, 39] directly take raw point clouds as input without converting them to other formats.”
Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View (2019, background): “HyperGraph (Bai, Zhang, and Torr 2019) Convolution&Attention FeaSt (Verma, Boyer, and Verbeek 2018) Convolution GraphSAGE (Hamilton, Ying, and Leskovec 2017) Convolution GAT (Veliˇckovi´c et al. 2018) Attention ARMA (Bianchi et al. 2019) Convolution GraphSAGE (Hamilton, Ying, and Leskovec 2017) Conv”
Fast end-to-end learning on protein surfaces (2020, background): “Followup works improved on these results using patch operators based on anisotropic diffusion (ACNN [10]), Gaussian mixtures (MoNet [29]), splines [17], graph message passing (FeastNet [41]), equivariant filters [31, 15], and primal-dual mesh operators [28].”
MeshCNN: a network with an edge (2019, background): “The local features that are encoded per point are influenced by the points residing in its Euclidean neighborhood.”
Graph convolutional networks: a comprehensive review (2019, background): “, [92, 93]), several graph convolutional network-based approaches have been proposed, including [5, 89].”
Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose (2020, methodology): “with simple stacked fully-connected layers. Wang et al. [63] adopted a GraphCNN to learn a deformation from an initial ellipsoid mesh to the target object mesh in a coarse-to-ne manner. Verma et al. [61] proposed a novel graph convolution operator and evaluated it on the shape correspondence problem”
Convolutional Mesh Regression for Single-Image Human Shape Reconstruction (2019, methodology): “[45] propose a new data-driven Graph Convolution operator with applications on shape analysis.”
PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding (2020, background): “Second, is the choice of neighborhood aggregation mechanism which could either be hierarchical [48, 33, 82, 16, 35], spatial CNN-like [30, 73, 37, 85, 59], spectral [78, 62, 65] or graph-based [72, 64, 68, 54].”
Dynamic Multiscale Graph Neural Networks for 3D Skeleton Based Human Motion Prediction (2020, background): “Graph deep learning: Graphs, expressing data associated with non-grid structures, preserve the dependencies among internal nodes [46, 40, 39].”
Learning to Dress 3D People in Generative Clothing (2019, background): “There is an extensive recent literature on methods and applications of graph convolutions [11, 25, 32, 41, 50].”
KPConv: Flexible and Deformable Convolution for Point Clouds (2019, background): “Despite the similarity between point convolutions and the most recent graph convolutions [39, 43], the latter learn filters on edge relationships instead of points relative positions.”
Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild (2020, background): “Alternatively, convolution operators can be defined with an attention mechanism to weight the neighbour selection [52, 51].”
Survey on Emotional Body Gesture Recognition (2018, background): “Richer, more realistic variants can be defined for example as a collection of connected cylinders or spheroids or 3D meshes [37], [38].”
Geometric Deep Learning for Volumetric Computational Fluid Dynamics (2019, methodology): “For completeness we also cite: the modern FeaStNet [25] which, under some restrictions, can be seen as a kind of MoNet with features-dependent learnable pseudo-coordinates 1; the MeshCNN [26] that operates directly on the edges of the mesh; and the SplineCNN [27] using continuous B-splines kernels.”