Mesh R-CNN

By Georgia Gkioxari, Jitendra Malik, Justin Johnson. 2019

Subfields: Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

Multiview Compressive Coding for 3D Reconstruction (2023, background): “This is an important finding as it suggests that expensive CAD supervision can be replaced with cheap RGB-D video captures.”
Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications (2022, background): “One highly related direction is to model and estimate 3D hand joints [5,44,45,53,60,63,66,72,78,79,80,85,3] and mesh [26,27,51,55,75,83,46], object pose [14,28,29,35,43,57,64,71,76,81], or both [6,17,18,19,69].”
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding (2020, methodology): “These datasets typically include rendered images, but do not include the underlying 3D assets used during rendering (e.g., triangle meshes), and are therefore not suitable for geometric learning problems that require direct 3D supervision (e.g., [27]).”
Bottleneck Transformers for Visual Recognition (2021, background): “Incorporating self-attention for other computer vision tasks such as keypoint detection [9] and 3D shape prediction [23]; studying self-attention architectures for self-supervised learning in computer vision [29, 26, 59, 11, 24, 12]; and scaling to much larger datasets such as JFT, YFCC and Instagra”
PointRend: Image Segmentation As Rendering (2019, background): “Instead, recent works consider more efﬁcient non-uniform representations such as meshes [42, 13], signed distance functions [32], and oc-trees [41].”
Atlas: End-to-End 3D Scene Reconstruction from Posed Images (2020, background): “hods encode the input to a small latent code and report results on single objects, mostly from shapenet [3]. As such it is not clear how to extend them to work on full scene reconstructions. MeshRCNN [16] builds upon 2D object detection [18] and adds an additional head to predict a voxel occupancy g”
Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild (2020, background): “There are several works [60,39,16] that recover the 3D shape of multiple objects but still reason about them independently.”
Accelerating 3D deep learning with PyTorch3D (2019, methodology): “Recently, Mesh R-CNN [14] shows that coarse voxel predictions capture instance-speciﬁc object topologies.”
MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion (2024, methodology): “Commonly used output representations include point clouds (Guo et al., 2020; Lin et al., 2018), meshes (Gkioxari et al., 2019; Wang et al., 2018), voxel (Sitzmann et al., 2019; Choy et al., 2016; Tulsiani et al., 2017), implicit representation (Wang et al., 2021a; Peng et al., 2020; Chen & Zhang,…”
Objaverse-XL: A Universe of 10M+ 3D Objects (2023, methodology): “Here, several methods explore novel representations [35, 36, 37, 38], network architectures [39, 40] and differentiable rendering techniques [41, 42, 43, 44, 45] to predict the 3D shapes and textures of objects from images with or without 3D supervision.”
Neural Unsigned Distance Fields for Implicit Function Learning (2020, background): “Alternatively, a shape can be approximated predicting by convexes directly [20] or combining voxels and meshes [28], but results are still coarse.”
Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation (2022, methodology): “3DRCNN [28] and Mesh-RCNN [16] takes as input a single RGB image and predicts 3D mesh and pose of object instances in the image.”
Reconstructing Hand-Object Interactions in the Wild (2020, background): “Some approaches [50, 21, 11, 22] work to predict the object shape, translation, and global orientation in the camera coordinate.”
Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction (2020, methodology): “Other works, such as Scan2Mesh [13] and Mesh-RCNN[20], use deep networks to predict meshes corresponding to range scans or RGB images, respectively.”
SynSin: End-to-End View Synthesis From a Single Image (2019, background): “Many works explore using a DNN to predict 3D object shapes [18, 20, 24, 26, 64, 69] or the depth of a scene given an image [6, 13, 33, 73].”
What's Behind the Couch? Directed Ray Distance Functions (DRDF) for 3D Scene Reconstruction (2021, unknown): “Most work on invisible surfaces focuses on single objects with voxels [10, 20, 23], point-clouds [16, 38], CAD models [29] and meshes [21, 22].”
Local Deep Implicit Functions for 3D Shape (2019, methodology): “Learned Shape Representations: To leverage shape priors, shape reconstruction methods began representing shape as a learned feature vector, with a trained decoder to a mesh [36, 14, 41, 15, 18], point cloud [10, 21, 46], voxel grid [9, 43, 5, 42], or octree [37, 33, 32].”
Video Self-Stitching Graph Network for Temporal Action Localization (2020, background): “As they become popular in different computer vision ﬁelds [11, 37, 39], researchers also ﬁnd their application in temporal action localization [42, 2, 44].”