Self-Supervised Visual Descriptor Learning for Dense Correspondence

By Tanner Schmidt, Richard A. Newcombe, Dieter Fox. 2017

172 citations recorded by the source corpus.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Self-Supervised Correspondence in Visuomotor Policy Learning (2019, background): “Other prior work [6] has used dynamic non-rigid reconstruction [35] to address dynamic scenes.”
Matterport3D: Learning from RGB-D Data in Indoor Environments (2017, background): “To enable training these deep descriptors, prior works leverage the vast amounts of correspondences found in existing RGB-D reconstruction datasets [29, 46].”
kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation (2019, neutral): “on is presented in Sec. IV. Other work has developed and used representations that may be more generalizable than object-speciﬁc pose estimation. Recent work has demonstrated dense visual descriptors [18] as a fully self-supervised object representation for manipulation that can generalize at the ca”
ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation (2024, background): “…it often requires both geometry and dynamics of environment to be modeled beforehand, various works have studied structural representations using data-driven methods, such as learning object-centric representation [19–34], particle-based dynamics [35–41], and keypoints or descriptors [3, 4, 42–54].”
SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Control (2018, background): “[15] learn robust visual descriptors for long-range associations using correspondences over short training sequences.”
Self-supervised 6D Object Pose Estimation for Robot Manipulation (2019, background): “Previous works on self-supervised learning in robot perception mainly focus on learning object segmentation [6], object detection [7], and dense pixel-wise correspondences [29], [30], since it is relatively easy to obtain annotations in 2D image space.”
Watching the World Go By: Representation Learning from Unlabeled Videos (2020, background): “Whereas self-supervised learning requires extra setup during data generation [9, 28, 31], unsupervised learning can use existing data without the need for any specific generation constraints.”
Graph-Structured Visual Imitation (2019, methodology): “We then minimize pixel-wise contrastive loss [39, 40], which forces matching pixels to be close in the learned feature spaces, while maintaining a distance margin for non-matching pixels.”
SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Planning and Control (2017, unknown): “Data association: Related work in the computer vision literature has looked at tackling the data association problem, primarily by matching visual descriptors, either hand-tuned [14], or more recently, learned using deep networks [15], [16].”
Visual Camera Re-Localization From RGB and RGB-D Images Using DSAC (2020, methodology): “Note that all RGB-D competitors, including the leading method, SS-VLD [6], models the uncertainty of scene coordinates in some form, i.e. predicting full distributions of image-to-scene correspondences.”
InLoc: Indoor Visual Localization with Dense Matching and View Synthesis (2019, background): “In contrast, indoor localization [11], [12], [45], [69], [71], [76], [85], [90] has received less attention in the last years.”
Mesh R-CNN (2019, background): “There is a broad line of work on multi-view reconstruction of objects and scenes, from classical binocular stereo [17, 53] to using shape priors [1, 2, 6, 21] and modern learning techniques [24, 26, 54].”
iNeRF: Inverting Neural Radiance Fields for Pose Estimation (2020, unknown): “[37, 45, 35]), because NeRF and iNeRF only require posed RGB images as training, iNeRF can be applied to localization as well.”
3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions (2016, methodology): “A more recent work [30] uses RGB-D reconstructions to train a 2D descriptor, while we train a 3D geometric descriptor.”
Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation (2018, methodology): “Relative to [7] we contribute distinct multi-object descriptors, new training procedures, a robotic automated collection pipeline, and demonstrate applications to robotic manipulation.”
FutureMapping: The Computational Structure of Spatial AI Systems (2018, methodology): “The machine learning methods which do this job will themselves improve by self-supervision during the SLAM process, taking advantage of dense SLAM’s properties as a ‘correspondence engine’ [50].”
State-of-the-Art Deep Learning: Evolving Machine Intelligence Toward Tomorrow’s Intelligent Network Traffic Control Systems (2017, background): “4) Robotics: In recent years, there has been a growing interest in using deep learning architectures in advanced robotics [87], [88].”